Utilities for new data processing pipeline

[odunbar]

Purpose

• Closes Add the utilites to perform rescaling, standardization and inverse-problem-aware dim reduction #364

Content

• Added a few typical univariate Affine scalings (min-max, quartile (aka robust), zscore)
• Added several multivariate decorrelations (to map samples to have N(0,I) distribution) or to whiten with structure matrices, or both. One can also reduce dimension to subspace,
• Added a CanonicalCorrelation analysis method. uses both input/output data to find a mutually reduced space for inputs and outputs
• Added tooling to extend with other encoders/decoders
• Added user interface to provide a schedule of several processing operations
• Unit tests for each encoder-decoder
• Added methods to encode new data (only) with a learnt schedule, or to decode data (only)
• Docstrings for the API

Example of use:

# for data:
    in_dim = 10
    out_dim = 50
    samples = 100
    prior_cov = ...
    in_data = rand(MvNormal(..., prior_cov), samples) 
    obs_noise_cov = ...
    out_data = rand(MvNormal(..., obs_noise_cov), samples) 
    io_pairs = PairedDataContainer(in_data, out_data)

User API to create a bunch of (silly) processing in order

    data_proc_schedule = [
        (zscore_scale(), "in_and_out"),  # univariate scaling of inputs and outputs with zscore
        (quartile_scale(), "in"), # scale inputs by univariate quartiles
        (decorrelate_sample_cov(), "out"), # transform outputs to subspace where the have distribution N(0,I)
        (minmax_scale(), "in_and_out"), # transform inputs and outputs so range is in [0,1]
    ]

pass into the emulator as (poss. with keywords)

Emulator(machine_learning_tool, io_pairs, prior_cov, obs_noise_cov, data_proc_schedule)

Then under the hood we perform

    # make schedule more parsable
    encoder_schedule = create_encoder_schedule(schedule_builder)
    # initialize encoders and encode
    enc_io_pairs, enc_prior_cov, enc_obs_noise_cov = 
        encode_with_schedule(encoder_schedule, io_pairs, prior_cov, obs_noise_cov)

### Displays:    
[ Info: Initialize encoding of data: "in" with AffineScaler: ZScoreScaling
[ Info: Initialize encoding of data: "out" with AffineScaler: ZScoreScaling
[ Info: Initialize encoding of data: "in" with AffineScaler: QuartileScaling
[ Info: Initialize encoding of data: "out" with Decorrelater: decorrelate_with=sample_cov
[ Info: Initialize encoding of data: "in" with AffineScaler: MinMaxScaling
[ Info: Initialize encoding of data: "out" with AffineScaler: MinMaxScaling

We can later decode all at once as

    dec_io_pairs, dec_prior_cov, dec_obs_noise_cov = decode_with_schedule(encoder_schedule, enc_io_pairs, enc_prior_cov, enc_obs_noise_cov)

We also have methods for using the schedule to encode new data and matrices easily etc. via:

id = DataContainer(new_in_data)
encoded_id = encode_with_schedule(encoder_schedule, id, "in")
decoded_id = decode_with_schedule(encoder_schedule, encoded_id, "in")
       
encoded_oc = encode_with_schedule(encoder_schedule, new_output_covariance, "out")
decoded_oc = decode_with_schedule(encoder_schedule, encoded_oc, "out")

---

• I have read and checked the items on the review checklist.

[codecov]

Codecov Report

Attention: Patch coverage is 98.33333% with 5 lines in your changes missing coverage. Please review.

Project coverage is 90.83%. Comparing base (4d36103) to head (93caaaa).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/Utilities.jl	98.33%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #363      +/-   ##
==========================================
+ Coverage   89.17%   90.83%   +1.65%     
==========================================
  Files           7        7              
  Lines        1358     1658     +300     
==========================================
+ Hits         1211     1506     +295     
- Misses        147      152       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ArneBouillon]

Looks good!

[ArneBouillon]

Aren't the diagnostic-based methods technically also affine?

[odunbar]

Yes - Ideas for a rename? Or maybe we could condense all the logic of the Diagnostic ones into this type too.

[ArneBouillon]

I liked your UnivariateScaler name suggestion, but others are fine too. After renaming, good to go for me!