Botorch preset by AdrianSosic · Pull Request #757 · emdgroup/baybe

AdrianSosic · 2026-03-02T15:14:00Z

DevPR, parent is #745

Adds the BOTORCH preset for GPs.

Important information

I think it's critical to actually assert that the preset exactly recovers the BoTorch behavior, in the form of a test, for mainly two reasons:
1. The construction involves quite a few things to be configured, i.e. handling both singletask/multitask (the latter even requiring a new custom gpytorch module), setting all sorts of priors correctly, etc. Blindly believing that everything is correct and then just claiming this is the BoTorch behavior seems like a bad idea. The test ensures this explicitly.
2. It also as an automatic alert mechanism for all situations when something is changed on the BoTorch side, informing us about breaking changes that yield different behavior but are not fully documented (which happened already several times).
I've also invested quite some effort to test the new multitask mean logic, i.e. that it not only recovers the BoTorch logic but that it also fills one of the missing gaps that will ultimately make our transfer learning model truely scale-invariant w.r.t. the different input tasks. In particular, I made sure that the only missing piece is the noise model stratification, which should be added in a follow-up PR and is the analogous to the stratification over means shipped by this PR. (This explains the changes to the streamlit script.)

Copilot

Pull request overview

Adds a new BOTORCH Gaussian Process preset intended to exactly mirror BoTorch GP behavior (single- and multi-task), alongside supporting kernel-factory refactors and validation improvements.

Changes:

Introduce GaussianProcessPreset.BOTORCH and implement BoTorch-aligned kernel / mean / likelihood factories (including custom GPyTorch components for multitask mean + likelihood).
Add parameter-kind classification (ParameterKind) and automatic parameter-kind validation for kernel factories via a new _KernelFactory base.
Extend tests (BoTorch parity + kernel-factory validation) and update the Streamlit surrogate-model demo to expose GP presets and multitask/transfer-learning toggles.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
`baybe/surrogates/gaussian_process/presets/botorch.py`	Adds BoTorch preset factories for kernel/mean/likelihood.
`baybe/surrogates/gaussian_process/components/_gpytorch.py`	Introduces custom GPyTorch mean + multitask likelihood helper to match BoTorch behavior.
`baybe/surrogates/gaussian_process/components/kernel.py`	Adds `_KernelFactory` w/ parameter-kind validation; adjusts `ICMKernelFactory`.
`baybe/surrogates/gaussian_process/presets/{edbo,edbo_smoothed}.py`	Migrates kernel factories to `_KernelFactory` and updates dimension logic.
`baybe/surrogates/gaussian_process/presets/{core,__init__.py}`	Adds `BOTORCH` enum entry and re-exports preset factories.
`baybe/parameters/{enum,base.py}`	Adds `ParameterKind` and exposes `Parameter.kind`.
`baybe/parameters/selectors.py`	Removes `_ParameterSelectorMixin` (superseded by `_KernelFactory`).
`tests/test_gp.py`	Adds parity test asserting `BOTORCH` preset reproduces BoTorch posterior stats.
`tests/test_kernel_factories.py`	Adds tests for kernel-factory parameter-kind validation.
`streamlit/surrogate_models.py`	Adds GP preset selector + multitask / transfer-learning UI and logic.
`docs/conf.py`	Updates nitpick ignore list for refactor.
`CHANGELOG.md`	Documents new presets and kernel-factory parameter-kind validation.

Comments suppressed due to low confidence (1)

baybe/surrogates/gaussian_process/presets/edbo.py:85

effective_dims is computed as train_x.shape[-1], which includes the task feature column in transfer-learning searchspaces. Since this factory explicitly excludes TaskParameter via parameter_selector, the priors should likely be based on the non-task dimensionality (i.e. exclude the task feature) to avoid skewing the dimension-dependent prior heuristics. Consider deriving effective_dims from the selected non-task dimensions instead (e.g., subtract task dims or count comp-rep columns for selected parameters).

        effective_dims = train_x.shape[-1]

        switching_condition = _contains_encoding(
            searchspace.discrete, _EDBO_ENCODINGS
        ) and (effective_dims >= 50)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

CLAassistant · 2026-04-24T10:47:48Z

All committers have signed the CLA.

AVHopp

The streamlit changes should be a separate PR as they have nothing to do with new presets. I do see the value in extending the streamlit but those two things should still not be combined imo.

AVHopp

Two bugs or uncatched errors in the streamlit

Scienfitz

sneaky global change to mll is not acceptable

Copilot

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

baybe/surrogates/gaussian_process/core.py:193

fit_criterion_factory changed its init alias from criterion_or_factory to fit_criterion_or_factory, which is a breaking change for any callers still using the old keyword. If this is intended, consider keeping criterion_or_factory as a deprecated alias (or otherwise providing a compatibility shim) so existing user code/configs don’t fail on upgrade.

    fit_criterion_factory: FitCriterionFactoryProtocol = field(
        alias="fit_criterion_or_factory",
        factory=BayBEFitCriterionFactory,
        converter=partial(  # type: ignore[misc]
            to_component_factory, component_type=GPComponentType.CRITERION
        ),
        validator=is_callable(),
    )

AVHopp

Minor things

Now properly handles the regular/task parameter split

AdrianSosic self-assigned this Mar 2, 2026

AdrianSosic added the new feature New functionality label Mar 2, 2026

AdrianSosic mentioned this pull request Mar 2, 2026

Beef up GaussianProcessSurrogate #745

Draft

28 tasks

AdrianSosic changed the base branch from main to dev/gp March 2, 2026 15:15

AdrianSosic added the dev label Mar 2, 2026

AdrianSosic force-pushed the feature/botorch_preset branch from a149bb3 to c3885f6 Compare March 3, 2026 08:34

AdrianSosic force-pushed the feature/botorch_preset branch from c3885f6 to 4f4fd55 Compare April 17, 2026 08:18

AdrianSosic marked this pull request as ready for review April 17, 2026 10:46

AdrianSosic requested review from AVHopp and Scienfitz as code owners April 17, 2026 10:46

Copilot AI review requested due to automatic review settings April 17, 2026 10:46

Copilot started reviewing on behalf of AdrianSosic April 17, 2026 10:46 View session

Copilot AI reviewed Apr 17, 2026

View reviewed changes

AdrianSosic force-pushed the feature/botorch_preset branch from 680d8a1 to eeaa63d Compare April 24, 2026 10:38

Copilot started work on behalf of AdrianSosic April 24, 2026 10:45 View session

Copilot finished work on behalf of AdrianSosic April 24, 2026 10:48

AdrianSosic force-pushed the feature/botorch_preset branch from e409228 to eeaa63d Compare April 24, 2026 10:51

AVHopp requested changes Apr 24, 2026

View reviewed changes

Comment thread CHANGELOG.md Outdated

Comment thread baybe/surrogates/gaussian_process/components/_gpytorch.py

Comment thread baybe/surrogates/gaussian_process/presets/botorch.py

Comment thread baybe/surrogates/gaussian_process/core.py Outdated

Comment thread streamlit/surrogate_models.py

AVHopp reviewed Apr 27, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/presets/botorch.py Outdated

AVHopp added this to the 0.15.0 milestone Apr 27, 2026

AdrianSosic force-pushed the feature/botorch_preset branch from eeaa63d to 7bb590b Compare April 27, 2026 15:45

AVHopp requested changes May 4, 2026

View reviewed changes

Comment thread streamlit/surrogate_models.py Outdated

Comment thread streamlit/surrogate_models.py

Comment thread streamlit/surrogate_models.py

Scienfitz requested changes May 4, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/presets/botorch.py Outdated

Comment thread tests/test_gp.py Outdated

Comment thread streamlit/surrogate_models.py

AdrianSosic force-pushed the feature/botorch_preset branch 6 times, most recently from 7ac4c25 to 9b90df2 Compare May 8, 2026 09:12

AdrianSosic added 7 commits May 18, 2026 09:07

Fix KernelFactory return types

15b10be

Make BotorchKernelFactory support parameter selection

46b96a4

Fix active dimensions validation

994e1a8

Bypass kernel warning for presets

5b312d9

Update CHANGELOG.md

95f2305

Rename on-task/off-task to target/source in streamlit

472d597

Fix missing fit_criterion_factory renamings

5be27f3

AdrianSosic force-pushed the feature/botorch_preset branch 3 times, most recently from 71b8c2b to 5c1f0c9 Compare May 18, 2026 09:31

AdrianSosic requested review from AVHopp, Scienfitz and Copilot May 18, 2026 09:32

Copilot started reviewing on behalf of AdrianSosic May 18, 2026 09:33 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/components/_gpytorch.py

Comment thread streamlit/surrogate_models.py Outdated

Comment thread baybe/surrogates/gaussian_process/presets/botorch.py Outdated

Comment thread baybe/surrogates/gaussian_process/presets/botorch.py Outdated

AVHopp approved these changes May 18, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/components/kernel.py

Comment thread baybe/surrogates/gaussian_process/presets/botorch.py

Comment thread baybe/surrogates/gaussian_process/presets/botorch.py Outdated

Scienfitz approved these changes May 18, 2026

View reviewed changes

Comment thread baybe/surrogates/gaussian_process/components/kernel.py

Comment thread baybe/surrogates/gaussian_process/presets/botorch.py Outdated

Comment thread tests/test_gp.py Outdated

AdrianSosic added 8 commits May 18, 2026 18:58

Fix dimension handling in BotorchKernelFactory

3050a6f

Now properly handles the regular/task parameter split

Add temporary ignore to pytest.ini

9c54d20

Fix deprecated .evaluate() call in test_kernels.py

873b8f6

Fix lazy imports

fbe2440

Fix dimension validation in ICMKernelFactory

fbe87ec

Fix kernel factory return types

0609e92

Drop duplicated kernel creation

f6f42a7

Fix vlines argument in streamlit script

7996e1e

AdrianSosic force-pushed the feature/botorch_preset branch from baa28f7 to 7996e1e Compare May 18, 2026 16:58

Hardwire MLL as criterion for BoTorch preset

2deed6c

AdrianSosic force-pushed the feature/botorch_preset branch from 4e2c202 to 2deed6c Compare May 19, 2026 14:54

AdrianSosic merged commit fd3fd21 into dev/gp May 20, 2026
12 of 13 checks passed

AdrianSosic deleted the feature/botorch_preset branch May 20, 2026 18:50

Conversation

AdrianSosic commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Important information

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

CLAassistant commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AVHopp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AVHopp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Scienfitz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AVHopp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

AdrianSosic commented Mar 2, 2026 •

edited

Loading

CLAassistant commented Apr 24, 2026 •

edited

Loading