Replacement of Emulator processing with new encoding framework

[odunbar]

Purpose

Closes #365

• Additional notes: The RF training is surprisingly sensitive to the processing, possibly priors for optimization? [UPDATE: Seems like it is not really the processing but has issues even without it]

Content

• Added the interface to use the encoder processors (else an empty processor, or don't provide one)
• Created a default that will decorrelate data, and use structure mat where possible

# if output_structure_matrix provided, but input not. 
# Default = 
encoder_schedule = [
     (decorrelate_sample_cov(), "in"),
     (decorrelate_structure_mat(), "out"),
]

• Replaced the encoding/decoding in the MCMC too
• Removed all the redundant old code that performed normalization etc.
• Add new ScalarRandomFeature and GaussianProcess regularization that can handle a diagonal but non-identity noise covariance - diagnosed from an output covariance if provided.
• Replaced in examples as below:
  • Emulators/Regression_2d_2d/
  • Emulators/Ishigami/
  • Emulators/G-function/
  • Emulators/L63/
  • Sinusoid/
  • Lorenz/ (and "Lorenz spatial dep")
  • Darcy/
  • Cloudy/
  • EDMF_data/ - not validated results
  • GCM/ - not validated results
• Updated all unit tests
• Added an initial documentation page

Misc

• Created issue Investigate/improve changes in RF performance under new toolings #368 for the RF dimension inputs and sensitivity to new dimension reduction
• Some examples needed a few additional modernizing changes
• New compatability with abstractGPs ("obsdim" keyword)

Replacement pattern in examples:

Before:

decorrelate = true # or false
Emulator(mlt, io_pairs; obs_noise_cov = A, decorrelate = bool1, standardize = bool2, retain_svd_frac=frac)

Replaced with something like

using CalibrateEmulateSample.Utilities
# examples..
encoder_schedule = nothing # default
encoder_schedule = [] # no processing
encoder_schedule = (decorrelate_structure_mat(retain_var = frac), "out")


Emulator(mlt, io_pairs; input_structure_matrix = A_in, output_structure_matrix = A_out, encoder_schedule = deepcopy(encoder_schedule) )

or with

Emulator(mlt, io_pairs; output_structure_matrix = A, encoder_schedule = deepcopy(encoder_schedule) )

Example of R2->R2 regression (GP-SKL) from 200 points

Currently no encoder performas similarly to encoding in this simple problem. The task is to Reconstruct the 2D outputs from training data shown (under dim1<->dim2 correlated by noise):

A few of the cases we have:

No processing (no-proc): mean and variance of prediction

Whiten with noise/prior struct-mat: mean and variance of prediction

Whiten then apply CCA whiten-then-cca: mean and variance of prediction

Example of Lorenz/emulate_sample_spatial_dep.jl

UQ with GP, and using the retain_var = 0.95, reduces R40->R80 map to R19->R40

---

• I have read and checked the items on the review checklist.

[ArneBouillon]

Are mutable structs an option for you?

[odunbar]

Unsure, I was told to not use them (a verry long time ago) but have not revisited. I think it is something that could be discussed at a later time however. Thanks for raising the question though and should not be horrible to change given we are adding more API for these structs

[ArneBouillon]

The commits since my merge look sensible to me :)