Ensembles with noisy realizations

[arrjon]

Feature description
In Ensemble allow for noisy realizations of the model by adding noise after simulating.

Motivation/Application
For example, When using Ensemble to get the posterior predictive we are currently neglecting the noise model. This is okay for prediction of the clean state. However, for checking if the uncertainty of the posterior matches the data we need to noise on top of the simulation (as noise could be different per simulation if the variance is estimated. I think it would be nice in general to have the option to get noisy realizations.

from pypesto.ensemble import Ensemble
from pypesto.predict import AmiciPredictor

ensemble = Ensemble.from_sample(
    result,
    x_names=problem.x_names,
    lower_bound=problem.lb,
    upper_bound=problem.ub,
)

predictor = AmiciPredictor(problem.objective)

ensemble_prediction = ensemble.predict(
    predictor,
    engine=pypesto.engine.MultiProcessEngine(),
)

# add assumed noise on top
for o in ensemble_prediction.prediction_results:
    o.conditions[0].output = o.conditions[0].output + np.random.normal(0, 0.05, size=len(o.conditions[0].output))

# Credible bands
axes = visualize.sampling_prediction_trajectories(
    ensemble_prediction,
    levels=[99, 95, 80],
    title='Posterior predictions',
)

[PaulJonasJost]

I like this, especially since uncertainty in an ensemble predicition comes from both sources (variance of parameters and of noise).

I would also welcome (for amici) a more easy-to-handle ensemble prediction, e.g., for the two default predictors (State and Observables), such that users can get an ensemble and an ensemble prediction with little code (akin to the predictor = which i think is currently actually much more complex).

[dilpath]

There are two options I see:

• if there are sufficient ensemble members, then adding noise to each observable trajectory then taking percentiles can be sufficient
• if there are insufficient ensemble members, we could construct a mixture model based on the noise model for each ensemble member observable trajectory, and take the percentiles of the mixture model. This would be exact I guess (and preferable to adding noise)