Settings and Skills for Claude. Initial application

[odunbar]

Purpose

• As with AI-maintainer skills wishlist EnsembleKalmanProcesses.jl#561 , This will implement and apply some initial AI tools
• Stem for skills copied from Add some initial Claude settings and skills. Apply these across the repo. EnsembleKalmanProcesses.jl#562

Content

• Create Claude settings.json - it encourages users to use skills and subagents. It prevents Claude using non-read-type git commands.
• Build some skills
  • build the following skills (using skill-creator from anthropic) from the wishlist:
    • base-show adds show and summary methods to structs
    • docstrings manages the docstring formatting and manages docs/src/API
    • error-message-manager manages the error messaging format, to ensure actionable, context-laden messages
• I then used these skills on the repository:
  • base-show was used to create show and summary methods for all structs. Includes both a compact and full show-type method (e.g. so that in arrays the compact 1-line show() method is used). unit tests are also written for this.
  • docstrings methods was used to overhaul the docstrings.
  • error-message-manager was used to remove @assert and update all error messages, it also adds test_throw error messages tested in the unit tests. Any duplicate errors, or those longer >3 line message is now put into a helper function _throw_xyz that are listed at the bottom of files. These have tests
  • The skills have a request to self-improve after different applications.

Examples

base-show

Neat base.show methods with some useful information and a compact 1-liner for vectors

julia> emulator
Emulator
  machine_learning_tool : GaussianProcess
  input_dim             : 8
  output_dim            : 49
  n_train               : 120 samples
  encoder (input)      : 8 → 8  Decorrelator
  encoder (output)     : 49 → 47  Decorrelator

julia> [emulator, emulator]
2-element Vector{Emulator{Float64, Vector{Any}}}:
 Emulator (GaussianProcess, 8→49)
 Emulator (GaussianProcess, 8→49)

julia> mcmc
MCMCWrapper
  prior_dim        : 8 parameters
  obs_dim          : 49
  n_obs            : 1 sample
  sampler          : RWMetropolisHastings
  encoder (input)  : 8 → 8  Decorrelator
  encoder (output): 49 → 47  Decorrelator

julia> (mcmc,mcmc)
(MCMCWrapper (8 params, RWMetropolisHastings), MCMCWrapper (8 params, RWMetropolisHastings))

docstrings

Struct and method definitions now more have structured formats. structs include recommended factories.

"""
Wrapper that stores an explicit forward map `f` in place of a trained `Emulator`.
When `predict` is called this object evaluates `E_out ∘ f ∘ c(x)`, where `c` is the
constraint transformation from the prior and `E_out` is the output encoder.

$(TYPEDEF)

# Fields

$(TYPEDFIELDS)

# Constructors

Prefer the [`forward_map_wrapper`](@ref) factory function for construction — it
builds and initialises the encoder schedule automatically from training data.

$(METHODLIST)
"""

"""
$(TYPEDSIGNATURES)

Construct an `Emulator` from a machine-learning tool and paired training data, fitting
the encoder schedule and building the underlying model in encoded space.

# Arguments

- `machine_learning_tool`: the `MachineLearningTool` to train (e.g. `GaussianProcess`, `ScalarRandomFeatureInterface`).
- `input_output_pairs`: training data as a `PairedDataContainer`.
- `encoder_schedule` (keyword, default `nothing`): encoding/decoding pipeline passed to
  [`create_encoder_schedule`](@ref). `nothing` builds a default schedule using
  [`decorrelate_sample_cov`](@ref) for both input and output, or
  `(decorrelate_sample_cov(), decorrelate_structure_mat())` when `:obs_noise_cov` is
  present in `encoder_kwargs`. Pass `[]` to disable encoding.
- `encoder_kwargs` (keyword, default `NamedTuple()`): forwarded to
  [`initialize_and_encode_with_schedule!`](@ref).
- Additional keywords are forwarded to the machine-learning tool initialiser.
"""

error-message-manager

Adds context to error messages, and helpers for longer/duplicated error messages

# inline 
# in predict(...ForwardMapWrapper,...)
size(new_inputs, 1) != check_dim &&
        _throw_input_dim_mismatch(size(new_inputs, 1), check_dim; caller = :ForwardMapWrapper)
# in predict(...Emulator...)
size(new_inputs, 1) != check_dim && _throw_input_dim_mismatch(size(new_inputs, 1), check_dim; caller = :Emulator)

# later
@noinline function _throw_input_dim_mismatch(actual::Int, expected::Int; caller::Symbol)
    throw(DimensionMismatch("""
$caller: new_inputs row count does not match the expected input dimension.

Expected:
    size(new_inputs, 1) == $expected

Got:
    size(new_inputs, 1) = $actual

Suggestion:
    Pass new_inputs with $expected rows (one per input dimension).
    If inputs are already in the encoded space, pass `encode = "in"` to predict(...).
"""))
end

These come with tests that test some message items, and the suggestion if it is concrete and actionable

let thrown = @test_throws DimensionMismatch EM.predict(fmw, x_test')
        @test contains(thrown.value.msg, "ForwardMapWrapper")
        @test contains(thrown.value.msg, "new_inputs row count")
end
# passing already-encoded inputs with encode = "in" must not throw
@test EM.predict(fmw, encode_data(fmw, x_test, "in"); encode = "in") isa Tuple

---

• I have read and checked the items on the review checklist.

[ArneBouillon]

Some great improvements!

[haakon-e]

LGTM!

One small thought: I've found empirically that when showing the contents of somewhat large structs in REPL, printing itself takes some time and is done line-by-line due to the pattern print(io, "some text"). I wonder if it's faster to create the full string, then print, but I have no evidence for this.

Anyhow, good to merge!

[odunbar]

@haakon-e interest point. I think where we are at, is that e.g. distributions and input-output pairs, or data samples needed for encoding were printed out in the past. I hope the show methods for EKP and CES help this 99% of the way from 1000+ line spams down to 5.

If I notice downstream issues from this we can look into solution.