diff --git a/AI_PACKAGE_SUMMARY.md b/AI_PACKAGE_SUMMARY.md index 06938cea2..fe29e40f9 100644 --- a/AI_PACKAGE_SUMMARY.md +++ b/AI_PACKAGE_SUMMARY.md @@ -7,7 +7,7 @@ This document explains how PlantSimEngine is structured internally, how models a PlantSimEngine is a Julia framework for composing plant models as modular processes. Users or modelers define models that implement a process, declare inputs/outputs, and optionally declare hard dependencies (manual calls). The engine builds a dependency graph (soft dependencies via inputs/outputs and hard dependencies via explicit model calls) and executes models in dependency order. It supports single-scale model lists and multiscale model mappings on a plant graph (MTG). Core modules (see `src/PlantSimEngine.jl`): -- `component_models`: `Status`, `RefVector`, `ModelList`, `TimeStepTable` +- `component_models`: `Status`, `RefVector`, `ModelMapping`, `TimeStepTable` - `dependencies`: dependency graph types and builders - `processes`: model interfaces, inputs/outputs/variables, process macro - `mtg`: multiscale mapping, GraphSimulation, initialization, save results @@ -29,7 +29,7 @@ File: `src/component_models/RefVector.jl` - Used for multiscale aggregation where a higher scale references values from many lower-scale nodes (e.g., plant-level model reads all leaves). - Updating a `RefVector` entry updates the referenced Status field. -### ModelList +### ModelList (deprecated) File: `src/component_models/ModelList.jl` - `ModelList` is the single-scale container: `models::NamedTuple`, `status::Status`, `dependency_graph::DependencyGraph`. - Building a ModelList: @@ -39,6 +39,7 @@ File: `src/component_models/ModelList.jl` - `type_promotion` can upcast default model values (not user-specified ones). ### MultiScaleModel + File: `src/mtg/MultiScaleModel.jl` - Wrapper to attach a multiscale variable mapping to a model. - Supports scalar mapping (SingleNode), vector mapping (MultiNode), renaming, and `PreviousTimeStep`. diff --git a/README.md b/README.md index 864091693..933958683 100644 --- a/README.md +++ b/README.md @@ -92,7 +92,7 @@ using PlantSimEngine using PlantSimEngine.Examples # Define the model: -model = ModelList( +model = ModelMapping( ToyLAIModel(), status=(TT_cu=1.0:2000.0,), # Pass the cumulated degree-days as input to the model ) @@ -136,7 +136,7 @@ lines(model[:TT_cu], model[:LAI], color=:green, axis=(ylabel="LAI (m² m⁻²)", ### Model coupling -Model coupling is done automatically by the package, and is based on the dependency graph between the models. To couple models, we just have to add them to the `ModelList`. For example, let's couple the `ToyLAIModel` with a model for light interception based on Beer's law: +Model coupling is done automatically by the package, and is based on the dependency graph between the models. To couple models, we just have to add them to the `ModelMapping`. For example, let's couple the `ToyLAIModel` with a model for light interception based on Beer's law: ```julia # ] add PlantSimEngine, DataFrames, CSV @@ -149,14 +149,14 @@ using PlantSimEngine.Examples meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) # Define the list of models for coupling: -model = ModelList( +model = ModelMapping( ToyLAIModel(), Beer(0.6), status=(TT_cu=cumsum(meteo_day[:, :TT]),), # Pass the cumulated degree-days as input to `ToyLAIModel`, this could also be done using another model ) ``` -The `ModelList` couples the models by automatically computing the dependency graph of the models. The resulting dependency graph is: +The `ModelMapping` couples the models by automatically computing the dependency graph of the models. The resulting dependency graph is: ``` ╭──── Dependency graph ──────────────────────────────────────────╮ @@ -223,7 +223,7 @@ fig The package is designed to be easily scalable, and can be used to simulate models at different scales. For example, you can simulate a model at the leaf scale, and then couple it with models at any other scale, *e.g.* internode, plant, soil, scene scales. Here's an example of a simple model that simulates plant growth using sub-models operating at different scales: ```julia -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( @@ -295,7 +295,7 @@ meteo = Weather( And run the simulation: ```julia -out_vars = Dict( +out_vars = ModelMapping( "Scene" => (:TT_cu,), "Plant" => (:carbon_allocation, :carbon_assimilation, :soil_water_content, :aPPFD, :TT_cu, :LAI), "Leaf" => (:carbon_demand, :carbon_allocation), diff --git a/benchmark/test-PSE-benchmark.jl b/benchmark/test-PSE-benchmark.jl index 3a7131d3b..0203fde34 100644 --- a/benchmark/test-PSE-benchmark.jl +++ b/benchmark/test-PSE-benchmark.jl @@ -27,16 +27,16 @@ function PlantSimEngine.run!(m::ToyInternodeCrazyEmergence, models, status, mete #end if length(MultiScaleTreeGraph.children(status.node)) == 1 && status.TT_cu - status.TT_cu_emergence >= m.TT_emergence - + status_new_internode = add_organ!(status.node, sim_object, "<", "Internode", 2, index=1) add_organ!(status_new_internode.node, sim_object, "+", "Leaf", 2, index=1) status_new_internode.TT_cu_emergence = status.TT_cu - elseif (length(MultiScaleTreeGraph.children(status.node)) >= 2 && length(MultiScaleTreeGraph.children(status.node)) < 7) && status.TT_cu - status.TT_cu_emergence >= m.TT_emergence + elseif (length(MultiScaleTreeGraph.children(status.node)) >= 2 && length(MultiScaleTreeGraph.children(status.node)) < 7) && status.TT_cu - status.TT_cu_emergence >= m.TT_emergence status_new_internode = add_organ!(status.node, sim_object, "<", "Internode", 2, index=1) add_organ!(status.node, sim_object, "+", "Leaf", 2, index=4) add_organ!(status.node, sim_object, "+", "Leaf", 2, index=5) status_new_internode.TT_cu_emergence = status.TT_cu - elseif (length(MultiScaleTreeGraph.children(status.node)) >= 7 && length(MultiScaleTreeGraph.children(status.node)) < 30) && status.TT_cu - status.TT_cu_emergence >= m.TT_emergence + elseif (length(MultiScaleTreeGraph.children(status.node)) >= 7 && length(MultiScaleTreeGraph.children(status.node)) < 30) && status.TT_cu - status.TT_cu_emergence >= m.TT_emergence add_organ!(status.node, sim_object, "+", "Leaf", 2, index=6) add_organ!(status.node, sim_object, "+", "Leaf", 2, index=7) add_organ!(status.node, sim_object, "+", "Leaf", 2, index=8) @@ -53,13 +53,13 @@ end # Wrapped this into a function so that it doesn't plague the benchmark with variables on a global scope #@check_allocs function do_benchmark_on_heavier_mtg() - mtg = import_mtg_example(); - + mtg = import_mtg_example() + # Example meteo, 365 timesteps : meteo_day = read_weather(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), duration=Day) - + #similar to the mtg growth test but with a much lower emergence threshold - mapping = Dict( + mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( @@ -110,13 +110,13 @@ function do_benchmark_on_heavier_mtg() ToySoilWaterModel(), ), ) - + out_vars = Dict( "Leaf" => (:carbon_assimilation, :carbon_demand, :soil_water_content, :carbon_allocation), "Internode" => (:carbon_allocation, :TT_cu_emergence), "Plant" => (:carbon_allocation,), "Soil" => (:soil_water_content,), ) - - out = run!(mtg, mapping, meteo_day, tracked_outputs=out_vars, executor=SequentialEx()); + + out = run!(mtg, mapping, meteo_day, tracked_outputs=out_vars, executor=SequentialEx()) end \ No newline at end of file diff --git a/benchmark/test-multirate-buffer-benchmark.jl b/benchmark/test-multirate-buffer-benchmark.jl index dc8b7f0a9..eb5a1cb9e 100644 --- a/benchmark/test-multirate-buffer-benchmark.jl +++ b/benchmark/test-multirate-buffer-benchmark.jl @@ -45,7 +45,7 @@ end function setup_multirate_buffer_benchmark(; nleaves=2000, ndays=30) mtg = _build_multirate_benchmark_mtg(nleaves) - mapping = Dict( + mapping = ModelMapping( "Leaf" => ( ModelSpec(MRBenchSourceModel(Ref(0))) |> TimeStepModel(1.0), ), diff --git a/benchmark/test-plantbiophysics.jl b/benchmark/test-plantbiophysics.jl index d9fd6817a..83b6f0c21 100644 --- a/benchmark/test-plantbiophysics.jl +++ b/benchmark/test-plantbiophysics.jl @@ -64,7 +64,7 @@ function benchmark_plantbiophysics() constants = Constants() #time_PB = Vector{Float64}(undef, N*microbenchmark_steps) for i = 1:N - leaf = ModelList( + leaf = ModelMapping( energy_balance=Monteith(), photosynthesis=Fvcb( VcMaxRef=set.VcMaxRef[i], @@ -138,9 +138,9 @@ function setup_benchmark_plantbiophysics_multitimestep() @. set[!, :vpd] = e_sat(set.T) - vapor_pressure(set.T, set.Rh) @. set[!, :aPPFD] = set.Ra_SW_f * 0.48 * 4.57 - leaf = Vector{ModelList}(undef, N) + leaf = Vector{ModelMapping}(undef, N) for i = 1:N - leaf[i] = ModelList( + leaf[i] = ModelMapping( energy_balance=Monteith(), photosynthesis=Fvcb( VcMaxRef=set.VcMaxRef[i], diff --git a/docs/src/FAQ/translate_a_model.md b/docs/src/FAQ/translate_a_model.md index 92f18fed8..4cfedbd97 100644 --- a/docs/src/FAQ/translate_a_model.md +++ b/docs/src/FAQ/translate_a_model.md @@ -146,7 +146,7 @@ meteo_day = to_daily(meteo, :TT => (x -> sum(x) / 24) => :TT) Then we can define our list of models, passing the values for `TT_cu` in the status at initialization: ```@example mymodel -m = ModelList( +m = ModelMapping( ToyLAIModel(), status = (TT_cu = cumsum(meteo_day.TT),), ) diff --git a/docs/src/developers.md b/docs/src/developers.md index b8ef8f2d1..09ca0e409 100644 --- a/docs/src/developers.md +++ b/docs/src/developers.md @@ -94,7 +94,7 @@ It’s probably now safe to request a merge. ### Other helpful things -⁃ In the `/PlantSimEngine/test` folder, there are a few basic helper functions. One of them outputs vectors of modellists, weather data, and output variables, which are used as a test bank/matrix for some tests, and provides wide coverage. If you wrote new models, new combinations of models, or added some new weather data, it helps to add them to the banks. +⁃ In the `/PlantSimEngine/test` folder, there are a few basic helper functions. One of them outputs vectors of ModelMapping, weather data, and output variables, which are used as a test bank/matrix for some tests, and provides wide coverage. If you wrote new models, new combinations of models, or added some new weather data, it helps to add them to the banks. ⁃ New downstream packages are worth adding to the integration and downstream package registry. ⁃ Unusual corner-cases are worth giving their own unit tests. Newly fixed bugs as well, even if the fix is fairly trivial. diff --git a/docs/src/index.md b/docs/src/index.md index d946933c6..5a41a9d4a 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -11,18 +11,18 @@ using PlantSimEngine.Examples # Import the example meteorological data: meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -# Define the model: -model = ModelList( - ToyLAIModel(), +# Define the model mapping: +model = ModelMapping( + ToyLAIModel(); status=(TT_cu=1.0:2000.0,), # Pass the cumulated degree-days as input to the model ) out = run!(model) -# Define the list of models for coupling: -model2 = ModelList( +# Define the mapping for coupled models: +model2 = ModelMapping( ToyLAIModel(), - Beer(0.6), + Beer(0.6); status=(TT_cu=cumsum(meteo_day[:, :TT]),), # Pass the cumulated degree-days as input to `ToyLAIModel`, this could also be done using another model ) out2 = run!(model2, meteo_day) @@ -118,9 +118,9 @@ using PlantSimEngine # Import the examples defined in the `Examples` sub-module using PlantSimEngine.Examples -# Define the model: -model = ModelList( - ToyLAIModel(), +# Define the model mapping: +model = ModelMapping( + ToyLAIModel(); status=(TT_cu=1.0:2000.0,), # Pass the cumulated degree-days as input to the model ) @@ -141,7 +141,7 @@ lines(out[:TT_cu], out[:LAI], color=:green, axis=(ylabel="LAI (m² m⁻²)", xla ### Model coupling -Model coupling is done automatically by the package, and is based on the dependency graph between the models. To couple models, we just have to add them to the `ModelList`. For example, let's couple the `ToyLAIModel` with a model for light interception based on Beer's law: +Model coupling is done automatically by the package, and is based on the dependency graph between the models. To couple models, we just have to add them to the `ModelMapping`. For example, let's couple the `ToyLAIModel` with a model for light interception based on Beer's law: ```@example readme # ] add PlantSimEngine, DataFrames, CSV @@ -153,10 +153,10 @@ using PlantSimEngine.Examples # Import the example meteorological data: meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -# Define the list of models for coupling: -model2 = ModelList( +# Define the mapping for coupled models: +model2 = ModelMapping( ToyLAIModel(), - Beer(0.6), + Beer(0.6); status=(TT_cu=cumsum(meteo_day[:, :TT]),), # Pass the cumulated degree-days as input to `ToyLAIModel`, this could also be done using another model ) @@ -164,7 +164,7 @@ model2 = ModelList( out2 = run!(model2, meteo_day) ``` -The `ModelList` couples the models by automatically computing the dependency graph of the models. The resulting dependency graph is: +The `ModelMapping` couples the models by automatically computing the dependency graph of the models. The resulting dependency graph is: ``` ╭──── Dependency graph ──────────────────────────────────────────╮ @@ -205,7 +205,7 @@ fig The package is designed to be easily scalable, and can be used to simulate models at different scales. For example, you can simulate a model at the leaf scale, and then couple it with models at any other scale, *e.g.* internode, plant, soil, scene scales. Here's an example of a simple model that simulates plant growth using sub-models operating at different scales: ```@example readme -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( @@ -295,8 +295,9 @@ We can then extract the outputs and convert them to a `DataFrame` for each scale ```@example readme using DataFrames -df_dict = convert_outputs(out, DataFrame) -sort!(df_dict["Leaf"], [:timestep, :node]) +df_outputs = convert_outputs(out, DataFrame) +leaf_df = df_outputs isa AbstractDict ? df_outputs["Leaf"] : df_outputs +sort!(leaf_df, [:timestep, :node]) ``` An example output of a multiscale simulation is shown in the documentation of PlantBiophysics.jl: diff --git a/docs/src/model_coupling/model_coupling_user.md b/docs/src/model_coupling/model_coupling_user.md index cb2d57dd1..a88471d9d 100644 --- a/docs/src/model_coupling/model_coupling_user.md +++ b/docs/src/model_coupling/model_coupling_user.md @@ -5,7 +5,7 @@ using PlantSimEngine, PlantMeteo # Import the example models defined in the `Examples` sub-module: using PlantSimEngine.Examples -m = ModelList( +m = ModelMapping( Process1Model(2.0), Process2Model(), Process3Model(), @@ -44,7 +44,7 @@ using PlantSimEngine.Examples Here is how we can make the model coupling: ```@example usepkg -m = ModelList(Process1Model(2.0), Process2Model(), Process3Model()) +m = ModelMapping(Process1Model(2.0), Process2Model(), Process3Model()) nothing # hide ``` @@ -75,7 +75,7 @@ outputs(Process2Model()) So considering those two models, we only need `var1` and `var2` to be initialized, as `var3` is computed. This is why we recommend [`to_initialize`](@ref) instead of [`inputs`](@ref), because it returns only the variables that need to be initialized, considering that some inputs are duplicated between models, and some are computed by other models (they are outputs of a model): ```@example usepkg -m = ModelList( +m = ModelMapping( Process1Model(2.0), Process2Model(), Process3Model(), @@ -88,7 +88,7 @@ to_initialize(m) The most straightforward way of initializing a model list is by giving the initializations to the `status` keyword argument during instantiation: ```@example usepkg -m = ModelList( +m = ModelMapping( Process1Model(2.0), Process2Model(), Process3Model(), @@ -118,7 +118,7 @@ All following models (`Process4Model` to `Process7Model`) do not call explicitly Let's make a new model list including the soft-coupled models: ```@example usepkg -m = ModelList( +m = ModelMapping( Process1Model(2.0), Process2Model(), Process3Model(), @@ -139,7 +139,7 @@ to_initialize(m) We can initialize it like so: ```@example usepkg -m = ModelList( +m = ModelMapping( Process1Model(2.0), Process2Model(), Process3Model(), diff --git a/docs/src/model_execution.md b/docs/src/model_execution.md index 93e84b601..1cfcd70f5 100644 --- a/docs/src/model_execution.md +++ b/docs/src/model_execution.md @@ -2,7 +2,7 @@ ## Simulation order -`PlantSimEngine.jl` uses the [`ModelList`](@ref) to automatically compute a dependency graph between the models and run the simulation in the correct order. When running a simulation with [`run!`](@ref), the models are then executed following this simple set of rules: +`PlantSimEngine.jl` uses the [`ModelMapping`](@ref) to automatically compute a dependency graph between the models and run the simulation in the correct order. When running a simulation with [`run!`](@ref), the models are then executed following this simple set of rules: 1. Independent models are run first. A model is independent if it can be run independently from other models, only using initializations (or nothing). 2. Then, models that have a dependency on other models are run. The first ones are the ones that depend on an independent model. Then the ones that are children of the second ones, and then their children ... until no children are found anymore. There are two types of children models (*i.e.* dependencies): hard and soft dependencies: @@ -93,7 +93,7 @@ aggregates over that civil day (including later timesteps from that day when ava ### Hold-last coupling (default policy) ```julia -mapping = Dict( +mapping = ModelMapping( "Leaf" => ( ModelSpec(LeafSourceModel()) |> TimeStepModel(1.0), ModelSpec(LeafConsumerModel()) |> @@ -106,7 +106,7 @@ mapping = Dict( ### Daily integration from hourly stream ```julia -mapping = Dict( +mapping = ModelMapping( "Leaf" => ( ModelSpec(HourlyAssimModel()) |> TimeStepModel(1.0), ), @@ -121,7 +121,7 @@ mapping = Dict( ### Interpolate slow producer to fast consumer ```julia -mapping = Dict( +mapping = ModelMapping( "Leaf" => ( ModelSpec(SlowSourceModel()) |> TimeStepModel(ClockSpec(2.0, 1.0)), ModelSpec(FastConsumerModel()) |> diff --git a/docs/src/multirate/multirate_tutorial.md b/docs/src/multirate/multirate_tutorial.md index 332c134a3..8f28c7bb8 100644 --- a/docs/src/multirate/multirate_tutorial.md +++ b/docs/src/multirate/multirate_tutorial.md @@ -105,7 +105,7 @@ leaf_proc = process(TutorialLeafHourlyModel()) plant_daily_proc = process(TutorialPlantDailyModel()) plant_weekly_proc = process(TutorialPlantWeeklyModel()) -mapping = Dict( +mapping = ModelMapping( "Leaf" => ( ModelSpec(TutorialLeafHourlyModel()) |> TimeStepModel(hourly) |> @@ -123,7 +123,6 @@ mapping = Dict( Ri_SW_q=(source=:Ri_SW_f, reducer=RadiationEnergy()), ) |> InputBindings(; leaf_assim_h=(process=leaf_proc, var=:leaf_assim_h, scale="Leaf", policy=Integrate())), - ModelSpec(TutorialPlantWeeklyModel()) |> ScopeModel(:plant) |> TimeStepModel(weekly) |> diff --git a/docs/src/multiscale/multiscale.md b/docs/src/multiscale/multiscale.md index 42973bb6d..b2d528672 100644 --- a/docs/src/multiscale/multiscale.md +++ b/docs/src/multiscale/multiscale.md @@ -24,7 +24,7 @@ It resembles the ToyAssimGrowth model used in the single-scale simulation [Model Our mapping between scale and model is therefore: ```@example usepkg -mapping = Dict("Leaf" => ToyAssimModel()) +mapping = ModelMapping("Leaf" => ToyAssimModel()) ``` Just like in single-scale simulations, we can call `to_initialize` to check whether variables need to be initialised. It will this time index by scale: @@ -38,7 +38,7 @@ In this example, the ToyAssimModel needs `:aPPFD` and `:soil_water_content` as i The initialization values for the variables can be passed along via a [`Status`](@ref) object: ```@example usepkg -mapping = Dict( +mapping = ModelMapping( "Leaf" => ( ToyAssimModel(), Status(aPPFD=1300.0, soil_water_content=0.5), @@ -61,7 +61,7 @@ It also makes sense to have that model operate at a different scale than the "Le ToyAssimModel is now makes use of the `soil_water_content` variable from the `"Soil"` scale, instead of at its own scale via the `Status` initialization. We therefore need to map `soil_water_content` from the "Soil" to the "Leaf" scale by wrapping `ToyAssimModel` in a `MultiScaleModel`: ```@example usepkg -mapping = Dict( +mapping = ModelMapping( "Soil" => ToySoilWaterModel(), "Leaf" => ( MultiScaleModel( @@ -91,7 +91,7 @@ Once again, `to_initialize` returns an empty dictionary, meaning the mapping is Let's now expand this mapping, to showcase other ways in which variables can be mapped from one scale to another. We'll keep the first two models, and add several more to simulate a couple of other processes within our plant. ```@example usepkg -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( @@ -144,17 +144,17 @@ nothing # hide This mapping might seem a little more daunting than previous examples, but several models should be recognizable in passing. In fact, you can consider this mapping to be an enhanced and more complex multi-scale version of a previous single-scale example, the coupling between photosynthesis model, a LAI model and a carbon biomass increment model, used in the [Model switching](@ref) subsection. ```julia -models2 = ModelList( +models2 = ModelMapping( ToyLAIModel(), Beer(0.5), - ToyAssimGrowthModel(), + ToyAssimGrowthModel(); status=(TT_cu=cumsum(meteo_day.TT),), ) ``` The multi-scale models simulate carbon capture via photosynthesis and carbon allocation for the plant organs' maintenance respiration and development. -The LAI and photosynthesis models are the same as in the ModelList example. The [`ToyDegreeDaysCumulModel`](@ref) provides the Cumulative Thermal Time to the plant. +The LAI and photosynthesis models are the same as in the single-scale mapping example. The [`ToyDegreeDaysCumulModel`](@ref) provides the Cumulative Thermal Time to the plant. The newly introduced models have the following dynamic : @@ -252,4 +252,4 @@ Or as a `DataFrame` dictionary using the [`DataFrames`](https://dataframes.julia ```@example usepkg using DataFrames df_dict = convert_outputs(outputs_sim, DataFrame) -``` \ No newline at end of file +``` diff --git a/docs/src/multiscale/multiscale_considerations.md b/docs/src/multiscale/multiscale_considerations.md index 7f2b1fe08..fd85f4145 100644 --- a/docs/src/multiscale/multiscale_considerations.md +++ b/docs/src/multiscale/multiscale_considerations.md @@ -11,8 +11,7 @@ Declaring and running a multi-scale simulation follows the same general workflow - a simulation requires a Multi-scale Tree Graph (MTG) to run and operates on that graph - when running, models are tied to a scale and only access local information -- models can run multiple times per timestep, -- the [`ModelList`](@ref) is replaced by a slightly more complex model mapping to link models to the scale they will operate at. +- models can run multiple times per timestep The simulation dependency graph will still be computed automatically and handle most couplings, meaning users don't need to specify the order of model execution once the extra code to declare the models is written. You will still need to declare hard dependencies, with extra considerations for multi-scale hard dependencies. @@ -22,7 +21,7 @@ Multi-scale simulations also tend to require more extra ad hoc models to prepare Other pages in the multiscale section describe : -- How to write a direct conversion of a single-scale ModelList simulation to a multi-scale simulation and add a second scale to it: [Converting a single-scale simulation to multi-scale](@ref), +- How convert a single-scale ModelMapping to a multi-scale one: [Converting a single-scale simulation to multi-scale](@ref), - A more complex multi-scale version of the single-scale simulation showcasing different variable mappings between scales: [Multi-scale variable mapping](@ref), - A three-part tutorial describing how to build up a combination of models to simulate a growing toy plant: [Writing a multiscale simulation](@ref), - Ways to handle situations where a variable ends up causing a cyclic dependency: [Avoiding cyclic dependencies](@ref), @@ -55,7 +54,7 @@ When users define which models they use, PlantSimEngine cannot determine in adva The user therefore needs to indicate for a simulation's which models are related to which scale. -A multi-scale mapping links models to the scale at which they operate, and is implemented as a Julia `Dict`, tying a scale, such as "Leaf" to models operating at that scale, such as "LeafSurfaceAreaModel". It is the equivalent of a [`ModelList`](@ref) in a single-scale simulation. +A multi-scale mapping links models to the scale at which they operate, and is also implemented in a [`ModelMapping`](@ref), tying a scale, such as "Leaf" to models operating at that scale, such as "LeafSurfaceAreaModel". Multi-scale models can be similar models to the ones found in earlier sections, or, if they need to make use of variables at other scales, may need to be wrapped as part of a [`MultiScaleModel`](@ref) object. Many models are not tied to a particular scale, which means those models can be reused at different scales or in single-scale simulations. @@ -76,10 +75,10 @@ This has two **important** consequences in terms of running a simulation : The [`run!`](@ref) function differs slightly from its single-scale version. The current structure (excluding a couple of advanced/deprecated kwargs) is the following: ```julia -run!(mtg, mapping, meteo, constants, extra; nsteps, tracked_outputs) +run!(mtg, mapping::ModelMapping, meteo, constants, extra; nsteps, tracked_outputs) ``` -Instead of a [`ModelList`](@ref), it takes an MTG and a mapping. The optional `meteo` and `constants` argument are identical to the single-scale version. The `extra` argument is now reserved and should not be used. A new `nsteps` keyword argument is available to restrict the simulation to a specified number of steps. +Instead of a just the [`ModelMapping`](@ref), it also takes an MTG as the first argument. The optional `meteo` and `constants` argument are identical to the single-scale version. The `extra` argument is now reserved and should not be used. A new `nsteps` keyword argument is available to restrict the simulation to a specified number of steps. ## Multi-scale output data structure @@ -152,7 +151,7 @@ Multi-scale simulations, especially for plants which have thousands of leaves, i Those tracked variables also need to be indexed by scale to avoid ambiguity: ```julia -outs = Dict( +outs = ModelMapping( "Scene" => (:TT, :TT_cu,), "Plant" => (:aPPFD, :LAI), "Leaf" => (:carbon_assimilation, :carbon_demand, :carbon_allocation, :TT), @@ -163,4 +162,4 @@ outs = Dict( ## Coupling and multi-scale hard dependencies -Multi-scale brings new types of coupling: mappings are part of the approach used to handle variables used by models at different scales. A model can also have a hard dependency on another model that operates at another scale. This multi-scale-specific complexity is discussed in [Handling dependencies in a multiscale context](@ref) \ No newline at end of file +Multi-scale brings new types of coupling: mappings are part of the approach used to handle variables used by models at different scales. A model can also have a hard dependency on another model that operates at another scale. This multi-scale-specific complexity is discussed in [Handling dependencies in a multiscale context](@ref) diff --git a/docs/src/multiscale/multiscale_coupling.md b/docs/src/multiscale/multiscale_coupling.md index 66934e88a..ce62af441 100644 --- a/docs/src/multiscale/multiscale_coupling.md +++ b/docs/src/multiscale/multiscale_coupling.md @@ -91,7 +91,7 @@ Here's a concrete example in [XPalm](https://github.com/PalmStudio/XPalm.jl), an The user-mapping includes the required models at specific organ levels. Here's the relevant portion of the mapping for the male reproductive organ : ```julia -mapping = Dict( +mapping = ModelMapping( ... "Male" => MultiScaleModel( diff --git a/docs/src/multiscale/multiscale_cyclic.md b/docs/src/multiscale/multiscale_cyclic.md index 30943188c..3dafb6053 100644 --- a/docs/src/multiscale/multiscale_cyclic.md +++ b/docs/src/multiscale/multiscale_cyclic.md @@ -10,7 +10,7 @@ For example the following mapping will raise an error: Example mapping ```julia - mapping_cyclic = Dict( + mapping_cyclic = ModelMapping( "Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), @@ -72,7 +72,7 @@ We can fix our previous mapping by computing the organs respiration using the ca !!! details ```@julia - mapping_nocyclic = Dict( + mapping_nocyclic = ModelMapping( "Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), diff --git a/docs/src/multiscale/multiscale_example_1.md b/docs/src/multiscale/multiscale_example_1.md index 23e76bb83..9f6c769b4 100644 --- a/docs/src/multiscale/multiscale_example_1.md +++ b/docs/src/multiscale/multiscale_example_1.md @@ -56,7 +56,7 @@ Let's also add a very artificial limiting factor: if the total leaf surface area We can expect the simulation mapping to look like a more complex version of the following: ```julia -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ToyStockComputationModel(), "Internode" => ToyCustomInternodeEmergence(), @@ -219,7 +219,7 @@ as opposed to the single-valued carbon stock mapped variable : And of course, some variables need to be initialized in the status: ```@example usepkg -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( diff --git a/docs/src/multiscale/multiscale_example_2.md b/docs/src/multiscale/multiscale_example_2.md index 4b33bb5e5..7815e07a4 100644 --- a/docs/src/multiscale/multiscale_example_2.md +++ b/docs/src/multiscale/multiscale_example_2.md @@ -197,7 +197,7 @@ The resource storage and internode emergence models now need a couple of extra w The "Root" organ is added to the mapping with its own models. New parameters need to be initialized. ```@example usepkg -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( diff --git a/docs/src/multiscale/multiscale_example_3.md b/docs/src/multiscale/multiscale_example_3.md index 5ab116d50..c0b4a4886 100644 --- a/docs/src/multiscale/multiscale_example_3.md +++ b/docs/src/multiscale/multiscale_example_3.md @@ -175,7 +175,7 @@ end PlantSimEngine.ObjectDependencyTrait(::Type{<:ToyLeafCarbonCaptureModel}) = PlantSimEngine.IsObjectIndependent() PlantSimEngine.TimeStepDependencyTrait(::Type{<:ToyLeafCarbonCaptureModel}) = PlantSimEngine.IsTimeStepIndependent() -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( @@ -416,7 +416,7 @@ end The new mapping only has straightforward changes. Some models cease to be multi-scale, others require new variables to be mapped for them. `carbon_root_creation_consumed` ceases to be a vector mapping and is a scalar variable. ```julia -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( @@ -471,7 +471,7 @@ The solution is hopefully quite intuitive : when we compute resource stocks, we The relevant part of the mapping that needs to be updated is the following: ```julia -mapping = Dict( +mapping = ModelMapping( ... "Plant" => ( MultiScaleModel( diff --git a/docs/src/multiscale/single_to_multiscale.md b/docs/src/multiscale/single_to_multiscale.md index 5f5ccb240..7643d0556 100644 --- a/docs/src/multiscale/single_to_multiscale.md +++ b/docs/src/multiscale/single_to_multiscale.md @@ -10,10 +10,10 @@ using CSV using DataFrames using MultiScaleTreeGraph meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -models_singlescale = ModelList( +models_singlescale = ModelMapping( ToyLAIModel(), Beer(0.5), - ToyRUEGrowthModel(0.2), + ToyRUEGrowthModel(0.2); status=(TT_cu=cumsum(meteo_day.TT),), ) ``` @@ -29,9 +29,9 @@ Pages = ["single_to_multiscale.md"] Depth = 3 ``` -# Converting the ModelList to a multi-scale mapping +# From single to multi-scale mapping -For example, let's return to the [`ModelList`](@ref) coupling a light interception model, a Leaf Area Index model, and a carbon biomass increment model that was discussed in the [Model switching](@ref) subsection: +For example, let's return to the [`ModelMapping`](@ref) coupling a light interception model, a Leaf Area Index model, and a carbon biomass increment model that was discussed in the [Model switching](@ref) subsection: ```@example usepkg using PlantMeteo @@ -41,10 +41,10 @@ using CSV meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -models_singlescale = ModelList( +models_singlescale = ModelMapping( ToyLAIModel(), Beer(0.5), - ToyRUEGrowthModel(0.2), + ToyRUEGrowthModel(0.2); status=(TT_cu=cumsum(meteo_day.TT),), ) @@ -53,10 +53,10 @@ outputs_singlescale = run!(models_singlescale, meteo_day) Those models all operate on a simplified model of a single plant, without any organ-local information. We can therefore consider them to be working at the 'whole plant' scale. Their variables also operate at that "plant" scale, so there is no need to map any variable to other scales. -We can therefore convert this into the following mapping : +We can therefore convert this into the following mapping: -```@example usepkg -mapping = Dict( +```@example usepkg +mapping = ModelMapping( "Plant" => ( ToyLAIModel(), Beer(0.5), @@ -65,7 +65,8 @@ mapping = Dict( ), ) ``` -Note the slight difference in syntax for the [`Status`](@ref). This is due to an implementation quirk (sorry). + +Note the slight difference in syntax for the [`Status`](@ref). This is because each scale has its own variables, so we must provide the values to each scale independently. ## Adding a new package for our plant graph @@ -84,9 +85,9 @@ mtg = MultiScaleTreeGraph.Node(MultiScaleTreeGraph.NodeMTG("/", "Plant", 0, 0),) We now have **almost** everything we need to run the multiscale simulation. -This first conversion step can be a starting point for a more elaborate multi-scale simulation. +This first conversion step can be a starting point for a more elaborate multi-scale simulation. -The signature of the [`run!`](@ref) function in multi-scale differs slightly from the ModelList version : +The signature of the [`run!`](@ref) function in multi-scale differs slightly from the single-scale version: ```julia out_multiscale = run!(mtg, mapping, meteo_day) @@ -94,17 +95,13 @@ out_multiscale = run!(mtg, mapping, meteo_day) (Some of the optional arguments also change slightly) -Unfortunately, there is one caveat. Passing in a vector through the [`Status`](@ref) field is still possible in multi-scale mode, but requires a little more advanced tinkering with the mapping, as it generates a custom model under the hood and the implementation is experimental and less user-friendly. - -If you are keen on going down that path, you can find a detailed example [here](@ref multiscale_vector), but we don't recommend it for beginners. - -What we'll do instead, is write our own model provide the thermal time per timestep as a variable, instead of as a single vector in the [`Status`](@ref). +Passing in a vector through the [`Status`](@ref) field is still possible in multi-scale mode, but more involving than in single-scale mode. If you need to go this way, you can find a detailed example [here](@ref multiscale_vector), although we don't recommend it for beginners. In any case, it's simpler to write a model to provide the thermal time per timestep as a variable, instead of as a single vector in the [`Status`](@ref). Our 'pseudo-multiscale' first approach will therefore turn into a genuine multi-scale simulation. ## Adding a second scale -Let's have a model provide the Cumulated Thermal Time to our Leaf Area Index model, instead of initializing it through the [`Status`](@ref). +Let's have a model provide the Cumulated Thermal Time to our Leaf Area Index model, instead of initializing it through the [`Status`](@ref). Let's instead implement our own `ToyTT_cuModel`. @@ -169,7 +166,7 @@ MultiScaleModel( and the new mapping with two scales: ```@example usepkg -mapping_multiscale = Dict( +mapping_multiscale = ModelMapping( "Scene" => ToyTt_CuModel(), "Plant" => ( MultiScaleModel( @@ -233,4 +230,4 @@ is_approx_equal = length(unique(computed_TT_cu_multiscale .≈ outputs_singlesca There is a model able to provide Thermal Time based on weather temperature data, [`ToyDegreeDaysCumulModel`](@ref), which can also be found in the examples folder. -We didn't make use of it here for learning purposes. It also computes a thermal time based on default parameters that don't correspond to the thermal time in the example weather data, so results differ from the thermal time already present in the weather data without tinkering with the parameters. \ No newline at end of file +We didn't make use of it here for learning purposes. It also computes a thermal time based on default parameters that don't correspond to the thermal time in the example weather data, so results differ from the thermal time already present in the weather data without tinkering with the parameters. diff --git a/docs/src/planned_features.md b/docs/src/planned_features.md index 440208264..382ed8dd1 100644 --- a/docs/src/planned_features.md +++ b/docs/src/planned_features.md @@ -33,7 +33,7 @@ Its current state doesn't enable practical declaration of several plant species, - Improved user errors - More examples - Better dependency graph traversal functions -- Ensure cyclic dependency checking and PreviousTimestep is active for ModelLists +- Ensure cyclic dependency checking and PreviousTimestep is active for ModelLists pathway ## Improvements on the testing side @@ -44,8 +44,8 @@ Its current state doesn't enable practical declaration of several plant species, ## Possible features (likely not a priority) -- API enabling iterative builds and validation of mappings and ModelLists -- Build step for the models, *i.e.* a function that would write a mapping or ModelList into a Julia script for validation, improved readability and (maybe) performance (no need to traverse the dependency graph anymore). +- API enabling iterative builds and validation of ModelMapping +- Build step for the models, *i.e.* a function that would write a ModelMapping into a Julia script for validation, improved readability and (maybe) performance (no need to traverse the dependency graph anymore). - Improved parallelisation - Reintroduce multi-object parallelisation in single-scale diff --git a/docs/src/prerequisites/installing_plantsimengine.md b/docs/src/prerequisites/installing_plantsimengine.md index 2c8f85bf8..33ef87535 100644 --- a/docs/src/prerequisites/installing_plantsimengine.md +++ b/docs/src/prerequisites/installing_plantsimengine.md @@ -68,7 +68,7 @@ Assuming you've setup you're environement, correctly added `PlantMeteo` and `Pla using PlantSimEngine, PlantMeteo using PlantSimEngine.Examples meteo = Atmosphere(T = 20.0, Wind = 1.0, Rh = 0.65, Ri_PAR_f = 500.0) -leaf = ModelList(Beer(0.5), status = (LAI = 2.0,)) +leaf = ModelMapping(Beer(0.5), status = (LAI = 2.0,)) out_sim = run!(leaf, meteo) ``` diff --git a/docs/src/prerequisites/key_concepts.md b/docs/src/prerequisites/key_concepts.md index 9021062fa..a2f077cd8 100644 --- a/docs/src/prerequisites/key_concepts.md +++ b/docs/src/prerequisites/key_concepts.md @@ -35,7 +35,7 @@ There may be different models that can be used for the same process; for instanc Models can also be used for ad hoc computations that aren't directly tied to a specific literature-defined physiological process. In PlantSimEngine, everything is a model. There are many instances where a custom model might be practical to aggregate some computations or handle other information. To illustrate, XPalm, the Oil Palm model, has a few models that handle the state of different organs, and a model to handle leaf pruning, which you can find [here](https://github.com/PalmStudio/XPalm.jl/blob/main/src/plant/phytomer/leaves/leaf_pruning.jl). -To prepare a simulation, you declare a ModelList with whatever models you wish to make use of and initialize necessary parameters: see the [step by step](@ref detailed-walkthrough-of-a-simple-simulation) section to learn how to use them in practice. +To prepare a simulation, you declare a ModelMapping with whatever models you wish to make use of and initialize necessary parameters: see the [step by step](@ref detailed-walkthrough-of-a-simple-simulation) section to learn how to use them in practice. For multi-scale simulations, models need to be tied to a particular scale when used. See the [Multiscale modeling](@ref) section below, or the [Multi-scale considerations](@ref) page for a more detailed description of multi-scale peculiarities. @@ -129,14 +129,10 @@ For example, a model of photosynthesis at the leaf scale can be combined with a When running multi-scale simulations which contain models operating at different organ levels for the plant, extra information needs to be provided by the user to run models. Since some models are reused at different organ levels, it is necessary to indicate which organ level a model operates at. -This is why multi-scale simulations make use of a 'mapping': the ModelList in the single-scale examples does not have a way to tie models to plant organs, and the more versatile models could be used in various places. The user must also indicate how models operate with other scales, *e.g.* if an input variable comes from another scale, it is required to indicate which scale it is mapped from. +This is why multi-scale simulations make use of a 'mapping' in the `ModelMapping`, as well as links between models/variables in different scales, *e.g.* if an input variable comes from another scale, it is required to indicate which scale it is mapped from. You can read more about some practical differences as a user between single- and multi-scale simulations here: [Multi-scale considerations](@ref). -!!! note - When you encounter the terms "Single-scale simulations", or "ModelList simulations", they will refer to simulations that are "not multi-scale". A multi-scale simulation makes use of a mapping between different organ/scale levels. A single-scale simulation has no such mapping, and uses the simpler ModelList interface. - You can implement a mapping that only makes use of a single scale level, of course, making it a "single-scale multi-scale simulation", but **unless otherwise specified, single-scale, and the whole section dedicated to single-scale simulations, refer to simulations with ModelList objects, and no mapping**. - ### Multi-scale Tree Graphs ![Grassy plant and equivalent MTG](../www/Grassy_plant_MTG_vertical.svg) diff --git a/docs/src/step_by_step/advanced_coupling.md b/docs/src/step_by_step/advanced_coupling.md index 36cad842b..622e4d0e2 100644 --- a/docs/src/step_by_step/advanced_coupling.md +++ b/docs/src/step_by_step/advanced_coupling.md @@ -5,7 +5,7 @@ using PlantSimEngine, PlantMeteo # Import the example models defined in the `Examples` sub-module: using PlantSimEngine.Examples -m = ModelList( +m = ModelMapping( Process1Model(2.0), Process2Model(), Process3Model(), @@ -43,7 +43,7 @@ end `Process2Model` is coupled to another process (`process1`), and calls its model's `run` function. The [`run!`](@ref) function is called with the same arguments as the [`run!`](@ref) function of the model that calls it, except that we pass the process we want to simulate as the first argument. !!! note - We don't enforce any type of model to simulate `process1`. This is the reason why we can switch so easily between model implementations for any process, by just changing the model in the [`ModelList`](@ref). + We don't enforce any type of model to simulate `process1`. This is the reason why we can switch so easily between model implementations for any process, by just changing the model in the [`ModelMapping`](@ref). A hard-dependency must always be declared to PlantSimEngine. This is done by adding a method to the `dep` function when implementing the model. For example, the hard-dependency to `process1` into `Process2Model` is declared as follows: diff --git a/docs/src/step_by_step/detailed_first_example.md b/docs/src/step_by_step/detailed_first_example.md index 6e38cca19..398ac61f7 100644 --- a/docs/src/step_by_step/detailed_first_example.md +++ b/docs/src/step_by_step/detailed_first_example.md @@ -10,7 +10,7 @@ If you simply wish to copy-paste examples and tinker with them, you can find a f using PlantSimEngine, PlantMeteo using PlantSimEngine.Examples meteo = Atmosphere(T = 20.0, Wind = 1.0, Rh = 0.65, Ri_PAR_f = 500.0) -leaf = ModelList(Beer(0.5), status = (LAI = 2.0,)) +leaf = ModelMapping(Beer(0.5), status = (LAI = 2.0,)) out_sim = run!(leaf, meteo) ``` @@ -31,7 +31,7 @@ A process in this package defines a biological or physical phenomena. Think of a A process is "declared", meaning we define a process, and then implement models for its simulation. In this example, we will make use of a process that was already defined, and for which there already is a model implementation. -### Models (ModelList) +### Models (ModelMapping) A process is simulated using a particular implementation, or **a model**. Each model is implemented using a structure that lists the parameters of the model. For example, PlantBiophysics provides the [`Beer`](https://vezy.github.io/PlantBiophysics.jl/stable/functions/#PlantBiophysics.Beer) structure for the implementation of the Beer-Lambert law of light extinction. The process of `light_interception` and the `Beer` model are provided as an example script in this package too at [`examples/Beer.jl`](https://github.com/VirtualPlantLab/PlantSimEngine.jl/blob/master/examples/Beer.jl). @@ -44,19 +44,19 @@ Models can use several types of entries: - Constants - Extras -**Parameters** are constant values that are used by the model to compute its outputs, and are exclusive to that model. +**Parameters** are constant values that are used by the model to compute its outputs, and are exclusive to that model. -**Meteorological information** contains values that are provided by the user and are used as inputs to the model. It is defined for one time-step, and `PlantSimEngine.jl` takes care of applying the model to each time-steps given by the user. +**Meteorological information** contains values that are provided by the user and are used as inputs to the model. It is defined for one time-step, and `PlantSimEngine.jl` takes care of applying the model to each time-steps given by the user. -**Variables** are either used or computed by the model and can optionally be initialized before the simulation. They can be part of multiple models, computed by one and then used as an input by another. They can also be a global simulation output, or be provided at the start of a simulation by the user. +**Variables** are either used or computed by the model and can optionally be initialized before the simulation. They can be part of multiple models, computed by one and then used as an input by another. They can also be a global simulation output, or be provided at the start of a simulation by the user. -**Constants** are constant values, usually common between models, *e.g.* the universal gas constant. +**Constants** are constant values, usually common between models, *e.g.* the universal gas constant. And **extras** are just extra values that can be used by a model, or serves as a placeholder for internal data. -Users declare a set of models used for simulation, as well as the necessary parameters for each model, and whatever variables need to be initialized. This is done using a [`ModelList`](@ref) structure. +Users declare a set of models used for simulation, as well as the necessary parameters for each model, and whatever variables need to be initialized. This is done using a [`ModelMapping`](@ref) structure. -For example let's instantiate a [`ModelList`](@ref) with a single model : the Beer-Lambert model of light extinction, used to simulate the light interception process. The model is implemented with the [`Beer`](https://github.com/VirtualPlantLab/PlantSimEngine.jl/blob/master/examples/Beer.jl) structure and only has one parameter: the extinction coefficient (`k`). +For example let's instantiate a [`ModelMapping`](@ref) with a single model : the Beer-Lambert model of light extinction, used to simulate the light interception process. The model is implemented with the [`Beer`](https://github.com/VirtualPlantLab/PlantSimEngine.jl/blob/master/examples/Beer.jl) structure and only has one parameter: the extinction coefficient (`k`). Importing the package: @@ -70,13 +70,13 @@ Import the examples defined in the [`Examples`](https://github.com/VirtualPlantL using PlantSimEngine.Examples ``` -And then declare a [`ModelList`](@ref) with the `Beer` model: +And then declare a [`ModelMapping`](@ref) with the `Beer` model: ```@example usepkg -m = ModelList(Beer(0.5)) +m = ModelMapping(Beer(0.5)) ``` -What happened here? We provided an instance of the `Beer` model to a [`ModelList`](@ref) to simulate the light interception process. +What happened here? We provided an instance of the `Beer` model to a [`ModelMapping`](@ref) to simulate the light interception process. ## Parameters @@ -88,7 +88,7 @@ fieldnames(Beer) ## Variables (inputs, outputs) -Variables are either inputs or outputs (*i.e.* computed) of models. Variables and their values are stored in the [`ModelList`](@ref) structure, and are initialized automatically or manually. +Variables are either inputs or outputs (*i.e.* computed) of models. Variables and their values are stored in the [`ModelMapping`](@ref) structure, and are initialized automatically or manually. For example, the `Beer` model needs the leaf area index (`LAI`, m² m⁻²) to run. @@ -104,17 +104,17 @@ and which are computed outputs of the model using [`outputs`](@ref): outputs(Beer(0.5)) ``` -The [`ModelList`](@ref) structure will keep track of every variable's current state when running the simulation, storing them in a field called `status`. We can inspect that field with the [`status`](@ref) function and see that in our example it has two variables: `LAI` and `PPFD`. The first is an input, the second an output (*i.e.* it is computed by the model). +The [`ModelMapping`](@ref) structure will keep track of every variable's current state when running the simulation, storing them in a field called `status`. We can inspect that field with the [`status`](@ref) function and see that in our example it has two variables: `LAI` and `PPFD`. The first is an input, the second an output (*i.e.* it is computed by the model). ```@example usepkg -m = ModelList(Beer(0.5)) +m = ModelMapping(Beer(0.5)) keys(status(m)) ``` To know which variables should be initialized, we can use [`to_initialize`](@ref): ```@example usepkg -m = ModelList(Beer(0.5)) +m = ModelMapping(Beer(0.5)) to_initialize(m) ``` @@ -127,18 +127,18 @@ Their values are uninitialized though (hence the warnings): Uninitialized variables are initialized to the value given in the [`inputs`](@ref) or [`outputs`](@ref) methods in the model's implementation code, which is usually equal to `typemin()`, *e.g.* `-Inf` for `Float64`. !!! tip - Prefer using [`to_initialize`](@ref) rather than [`inputs`](@ref) to check which variables should be initialized. [`inputs`](@ref) returns every variable that is needed by the model to run, but in multi-model simulations, some of them may already be computed by other models and not require initialization. [`to_initialize`](@ref) returns **only** the variables that are needed by the model to run and that are not initialized in the [`ModelList`](@ref). + Prefer using [`to_initialize`](@ref) rather than [`inputs`](@ref) to check which variables should be initialized. [`inputs`](@ref) returns every variable that is needed by the model to run, but in multi-model simulations, some of them may already be computed by other models and not require initialization. [`to_initialize`](@ref) returns **only** the variables that are needed by the model to run and that are not initialized in the [`ModelMapping`](@ref). -We can initialize the required variables by providing their starting values to the status when declaring the `ModelList`: +We can initialize the required variables by providing their starting values to the status when declaring the `ModelMapping`: ```@example usepkg -m = ModelList(Beer(0.5), status = (LAI = 2.0,)) +m = ModelMapping(Beer(0.5), status = (LAI = 2.0,)) ``` Or after instantiation using [`init_status!`](@ref): ```@example usepkg -m = ModelList(Beer(0.5)) +m = ModelMapping(Beer(0.5)) init_status!(m, LAI = 2.0) ``` @@ -172,7 +172,7 @@ More details are available from the [package documentation](https://vezy.github. ### Simulation of processes -To run a simulation, you can call the [`run!`](@ref) method on the [`ModelList`](@ref). If some meteorological data is required for models to be simulated over several timesteps, that can be passed in as an optional argument as well. +To run a simulation, you can call the [`run!`](@ref) method on the [`ModelMapping`](@ref). If some meteorological data is required for models to be simulated over several timesteps, that can be passed in as an optional argument as well. Your call to the function would then look like this: @@ -180,9 +180,9 @@ Your call to the function would then look like this: run!(model_list, meteo) ``` -The first argument is the model list (see [`ModelList`](@ref)), and the second defines the micro-climatic conditions. +The first argument is the model mapping (see [`ModelMapping`](@ref)), and the second defines the micro-climatic conditions. -The [`ModelList`](@ref) should already be initialized for the given process before calling the function. Refer to the earlier subsection [Variables (inputs, outputs)](@ref) for more details. +The [`ModelMapping`](@ref) should already be initialized for the given process before calling the function. Refer to the earlier subsection [Variables (inputs, outputs)](@ref) for more details. ### Example simulation @@ -196,7 +196,7 @@ using PlantSimEngine.Examples meteo = Atmosphere(T = 20.0, Wind = 1.0, Rh = 0.65, Ri_PAR_f = 500.0) -leaf = ModelList(Beer(0.5), status = (LAI = 2.0,)) +leaf = ModelMapping(Beer(0.5), status = (LAI = 2.0,)) outputs_example = run!(leaf, meteo) @@ -205,12 +205,11 @@ outputs_example[:aPPFD] ### Outputs -The [`status`](@ref) field of a [`ModelList`](@ref) is used to initialize the variables before simulation and then to keep track of their values during and after the simulation. We can extract outputs of the very last timestep of a simulation using the [`status`](@ref) function. +The [`status`](@ref) field of a [`ModelMapping`](@ref) is used to initialize the variables before simulation and then to keep track of their values during and after the simulation. We can extract outputs of the very last timestep of a simulation using the [`status`](@ref) function. The actual full output data is returned by the [`run!`](@ref) function. Data is usually stored in a [`TimeStepTable`](@ref) structure from `PlantMeteo.jl`, which is a fast DataFrame-like structure with each time step being a [`Status`](@ref). It can be also be any `Tables.jl` structure, such as a regular `DataFrame`. The weather is also usually stored in a [`TimeStepTable`](@ref) but with each time step being an `Atmosphere`. -In our example, the simulation was only provided one weather timestep, so the outputs returned by [`run!`](@ref) and the ModelList's [`status`](@ref) field are identical. - +In our example, the simulation was only provided one weather timestep, so the outputs returned by [`run!`](@ref) and the ModelMapping's [`status`](@ref) field are identical. Let's look at the outputs structure of our previous simulated leaf: ```@setup usepkg diff --git a/docs/src/step_by_step/implement_a_model.md b/docs/src/step_by_step/implement_a_model.md index 5cc143a43..afaf888f3 100644 --- a/docs/src/step_by_step/implement_a_model.md +++ b/docs/src/step_by_step/implement_a_model.md @@ -170,14 +170,14 @@ These functions are internal, and end with an "\_". Users instead use [`inputs`] ### The run! method -When running a simulation with [`run!`](@ref), each model is run in turn at every timestep, following whatever order was deduced from the ModelList definition and Status. Each model also has its [`run!`](@ref) method for that purpose that update the simulation's current state, with a slightly different signature. The function takes six arguments: +When running a simulation with [`run!`](@ref), each model is run in turn at every timestep, following whatever order was deduced from the `ModelMapping` definition and Status. Each model also has its [`run!`](@ref) method for that purpose that update the simulation's current state, with a slightly different signature. The function takes six arguments: ```julia function run!(::Beer, models, status, meteo, constants, extras) ``` - the model's type -- models: a [`ModelList`](@ref) object, which contains all the models of the simulation +- models: a [`ModelMapping`](@ref) object, which contains all the models of the simulation - status: a [`Status`](@ref) object, which contains the current values (*i.e.* state) of the variables for **one** time-step (e.g. the value of the plant LAI at time t) - meteo: (usually) an `Atmosphere` object, or a row of the meteorological data, which contains the current values of the meteorological variables for **one** time-step (*e.g.* the value of the PAR at time t) - constants: a `Constants` object, or a `NamedTuple`, which contains the values of the constants for the simulation (*e.g.* the value of the Stefan-Boltzmann constant, unit-conversion constants...) @@ -185,7 +185,7 @@ function run!(::Beer, models, status, meteo, constants, extras) A typical [`run!`](@ref) function can therefore make use of simulation constants, input/output variables accessible through the [`Status`](@ref object, or weather data. -Here is the [`run!`](@ref) implementation of the light interception for a [`ModelList`](@ref) component models. Note that the input and output variable are accessed through the [`status`](@ref) argument : +Here is the [`run!`](@ref) implementation of the light interception for a [`ModelMapping`](@ref) component models. Note that the input and output variable are accessed through the [`status`](@ref) argument : ```@example usepkg function run!(::Beer, models, status, meteo, constants, extras) @@ -203,7 +203,7 @@ To use this model, users will have to make sure that the variables for that mode !!! Note [`Status`](@ref) objects contain the current state of the simulation. It is not, by default, possible to make use of earlier variable states, unless a custom model is written for that purpose. -Model parameters are available from the [`ModelList`](@ref) that is passed via the `models` argument. Index by the process name, then the parameter name. For example, the `k` parameter of the `Beer` model is found in `models.light_interception.k`. +Model parameters are available from the [`ModelMapping`](@ref) that is passed via the `models` argument. Index by the process name, then the parameter name. For example, the `k` parameter of the `Beer` model is found in `models.light_interception.k`. !!! warning You need to import all the functions you want to extend, so Julia knows your intention of adding a method to the function from PlantSimEngine, and not defining your own function. To do so, you have to prefix the said functions by the package name, or import them before *e.g.*: `import PlantSimEngine: inputs_, outputs_`. The troubleshooting subsection [Implementing a model: forgetting to import or prefix functions](@ref) showcases output errors that can occur when you forget to prefix. diff --git a/docs/src/step_by_step/model_switching.md b/docs/src/step_by_step/model_switching.md index 4442007b0..b091351f7 100644 --- a/docs/src/step_by_step/model_switching.md +++ b/docs/src/step_by_step/model_switching.md @@ -7,14 +7,14 @@ using PlantSimEngine.Examples meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -models = ModelList( +models = ModelMapping( ToyLAIModel(), Beer(0.5), ToyRUEGrowthModel(0.2), status=(TT_cu=cumsum(meteo_day.TT),), ) run!(models, meteo_day) -models2 = ModelList( +models2 = ModelMapping( ToyLAIModel(), Beer(0.5), ToyAssimGrowthModel(), @@ -25,11 +25,11 @@ run!(models2, meteo_day) One of the main objective of PlantSimEngine is allowing users to switch between model implementations for a given process **without making any change to the PlantSimEngine codebase**. -The package was designed around this idea to make easy changes easy and efficient. Switch models in the [`ModelList`](@ref), and call the [`run!`](@ref) function again. No other changes are required if no new variables are introduced. +The package was designed around this idea to make easy changes easy and efficient. Switch models in the [`ModelMapping`](@ref), and call the [`run!`](@ref) function again. No other changes are required if no new variables are introduced. ## A first simulation as a starting point -With a working environment, let's create a [`ModelList`](@ref) with several models from the example scripts in the [`examples`](https://github.com/VirtualPlantLab/PlantSimEngine.jl/blob/master/examples/) folder: +With a working environment, let's create a [`ModelMapping`](@ref) with several models from the example scripts in the [`examples`](https://github.com/VirtualPlantLab/PlantSimEngine.jl/blob/master/examples/) folder: Importing the models from the scripts: @@ -39,10 +39,10 @@ using PlantSimEngine using PlantSimEngine.Examples ``` -Coupling the models in a [`ModelList`](@ref): +Coupling the models in a [`ModelMapping`](@ref): ```@example usepkg -models = ModelList( +models = ModelMapping( ToyLAIModel(), Beer(0.5), ToyRUEGrowthModel(0.2), @@ -67,12 +67,12 @@ output_initial = run!(models, meteo_day) ## Switching one model in the simulation -Now what if we want to switch the model that computes growth ? We can do this by simply replacing the model in the [`ModelList`](@ref), and PlantSimEngine will automatically update the dependency graph, and adapt the simulation to the new model. +Now what if we want to switch the model that computes growth ? We can do this by simply replacing the model in the [`ModelMapping`](@ref), and PlantSimEngine will automatically update the dependency graph, and adapt the simulation to the new model. Let's switch ToyRUEGrowthModel with ToyAssimGrowthModel: ```@example usepkg -models2 = ModelList( +models2 = ModelMapping( ToyLAIModel(), Beer(0.5), ToyAssimGrowthModel(), # This was `ToyRUEGrowthModel(0.2)` before @@ -93,7 +93,7 @@ output_updated = run!(models2, meteo_day) And that's it! We can switch between models without changing the code, and without having to recompute the dependency graph manually. This is a very powerful feature of PlantSimEngine!💪 !!! note - This was a very standard but straightforward example. Sometimes other models will require to add other models to the [`ModelList`](@ref). For example ToyAssimGrowthModel could have required a maintenance respiration model. In this case `PlantSimEngine` will indicate what kind of model is required for the simulation. + This was a very standard but straightforward example. Sometimes other models will require to add other models to the [`ModelMapping`](@ref). For example ToyAssimGrowthModel could have required a maintenance respiration model. In this case `PlantSimEngine` will indicate what kind of model is required for the simulation. !!! note In our example we replaced what we call a [soft-dependency coupling](@ref hard_dependency_def), but the same principle applies to [hard-dependencies](@ref hard_dependency_def). Hard and Soft dependencies are concepts related to model coupling, and are discussed in more detail in [Standard model coupling](@ref) and [Coupling more complex models](@ref). diff --git a/docs/src/step_by_step/quick_and_dirty_examples.md b/docs/src/step_by_step/quick_and_dirty_examples.md index 8aa8d7405..e8a022e98 100644 --- a/docs/src/step_by_step/quick_and_dirty_examples.md +++ b/docs/src/step_by_step/quick_and_dirty_examples.md @@ -25,7 +25,7 @@ These examples assume you have a working Julia environment with PlantSimengine a using PlantSimEngine, PlantMeteo using PlantSimEngine.Examples meteo = Atmosphere(T = 20.0, Wind = 1.0, Rh = 0.65, Ri_PAR_f = 500.0) -leaf = ModelList(Beer(0.5), status = (LAI = 2.0,)) +leaf = ModelMapping(Beer(0.5), status = (LAI = 2.0,)) out = run!(leaf, meteo) ``` @@ -41,7 +41,7 @@ using PlantSimEngine.Examples meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -models = ModelList( +models = ModelMapping( ToyLAIModel(), Beer(0.5), status=(TT_cu=cumsum(meteo_day.TT),), @@ -61,7 +61,7 @@ using PlantSimEngine.Examples meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -models = ModelList( +models = ModelMapping( ToyLAIModel(), Beer(0.5), ToyRUEGrowthModel(0.2), @@ -84,7 +84,7 @@ using PlantBiophysics, PlantSimEngine meteo = Atmosphere(T = 22.0, Wind = 0.8333, P = 101.325, Rh = 0.4490995) -leaf = ModelList( +leaf = ModelMapping( Monteith(), Fvcb(), Medlyn(0.03, 12.0), diff --git a/docs/src/step_by_step/simple_model_coupling.md b/docs/src/step_by_step/simple_model_coupling.md index 548776f7e..70f1069c8 100644 --- a/docs/src/step_by_step/simple_model_coupling.md +++ b/docs/src/step_by_step/simple_model_coupling.md @@ -6,7 +6,7 @@ using PlantSimEngine.Examples using CSV using DataFrames meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -models = ModelList( +models = ModelMapping( ToyLAIModel(), Beer(0.5), ToyRUEGrowthModel(0.2), @@ -19,24 +19,24 @@ nothing Again, make sure you have a working Julia environment with PlantSimengine added to it, and the other recommended companion packages. Details for getting to that point are provided on the [Installing and running PlantSimEngine](@ref) page. -## ModelList +## ModelMapping -The [`ModelList`](@ref) is a container that holds a list of models, their parameter values, and the status of the variables associated to them. +The [`ModelMapping`](@ref) is a container that holds a list of models, their parameter values, and the status of the variables associated to them. -If one looks at prior examples, the Modellists so far have only contained a single model, whose input variables are initialised in the Modellist [`status`](@ref) keyword argument. +If one looks at prior examples, the ModelMappings so far have only contained a single model, whose input variables are initialised in the ModelMapping [`status`](@ref) keyword argument. Example models are all taken from the example scripts in the [`examples`](https://github.com/VirtualPlantLab/PlantSimEngine.jl/blob/master/examples/) folder. -Here's a first [`ModelList`](@ref) declaration with a light interception model, requiring input Leaf Area Index (LAI): +Here's a first [`ModelMapping`](@ref) declaration with a light interception model, requiring input Leaf Area Index (LAI): ```julia -modellist_coupling_part_1 = ModelList(Beer(0.5), status = (LAI = 2.0,)) +modellist_coupling_part_1 = ModelMapping(Beer(0.5), status = (LAI = 2.0,)) ``` Here's a second one with a Leaf Area Index model, with some example Cumulated Thermal Time as input. (This TT_cu is usually computed from weather data): ```julia -modellist_coupling_part_2 = ModelList( +modellist_coupling_part_2 = ModelMapping( ToyLAIModel(), status=(TT_cu=1.0:2000.0,), # Pass the cumulated degree-days as input to the model ) @@ -46,7 +46,7 @@ modellist_coupling_part_2 = ModelList( Suppose we want our ToyLAIModel to compute the `LAI` for the light interception model. -We can couple the two models by having them be part of a single [`ModelList`](@ref). The `LAI` variable will then be a coupled output computed by the ToyLAIModel, then used as input by `Beer`. It will no longer need to be declared as part of the [`status` . +We can couple the two models by having them be part of a single [`ModelMapping`](@ref). The `LAI` variable will then be a coupled output computed by the ToyLAIModel, then used as input by `Beer`. It will no longer need to be declared as part of the [`status` . This is an instance of what we call a ["soft dependency" coupling](@ref hard_dependency_def): a model depends on another model's outputs for its inputs. @@ -57,8 +57,8 @@ using PlantSimEngine # Import the examples defined in the `Examples` sub-module: using PlantSimEngine.Examples -# A ModelList with two coupled models -models = ModelList( +# A ModelMapping with two coupled models +models = ModelMapping( ToyLAIModel(), Beer(0.5), status=(TT_cu=1.0:2000.0,), @@ -97,8 +97,8 @@ using PlantSimEngine.Examples # Import example weather data meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -# A ModelList with two coupled models -models = ModelList( +# A ModelMapping with two coupled models +models = ModelMapping( ToyLAIModel(), Beer(0.5), status=(TT_cu=cumsum(meteo_day.TT),), # We can now compute a genuine cumulative thermal time from the weather data @@ -113,10 +113,10 @@ And there you have it. The light interception model made its computations using ## Further coupling -Of course, one can keep adding models. Here's an example ModelList with another model, ToyRUEGrowthModel, which computes the carbon biomass increment caused by photosynthesis. +Of course, one can keep adding models. Here's an example `ModelMapping` with another model, `ToyRUEGrowthModel`, which computes the carbon biomass increment caused by photosynthesis. ```julia -models = ModelList( +models = ModelMapping( ToyLAIModel(), Beer(0.5), ToyRUEGrowthModel(0.2), diff --git a/docs/src/troubleshooting_and_testing/plantsimengine_and_julia_troubleshooting.md b/docs/src/troubleshooting_and_testing/plantsimengine_and_julia_troubleshooting.md index 7a2be1c95..131edefae 100644 --- a/docs/src/troubleshooting_and_testing/plantsimengine_and_julia_troubleshooting.md +++ b/docs/src/troubleshooting_and_testing/plantsimengine_and_julia_troubleshooting.md @@ -99,10 +99,10 @@ The syntax for an empty NamedTuple is `NamedTuple()`. If instead one types `()` Most of the following errors occur exclusively in multi-scale simulations, which has a slightly more complex API, but some are common to both single- and multi-scale simulations. -### ModelList/Mapping : providing a type name instead of an constructed instance +### ModelMapping: providing a type name instead of a constructed instance ```julia -m = ModelList(day=MyToyModel, week=MyToyModel2) +m = ModelMapping(day=MyToyModel, week=MyToyModel2) ``` This line is incorrect and will return ```julia @@ -111,7 +111,7 @@ MethodError: no method matching inputs_(::Type{MyToyDayModel}) The correct syntax is (assuming the corresponding constructor exists) : ```julia -m = ModelList(day=MyToyModel(), week=MyToyModel2()) +m = ModelMapping(day=MyToyModel(), week=MyToyModel2()) ``` ### Implementing a model: forgetting to import or prefix functions @@ -148,7 +148,7 @@ meteo = Weather([ Atmosphere(T=18.0, Wind=1.0, Rh=0.65, Ri_PAR_f=100.0), ]) -model = ModelList( +model = ModelMapping( ToyToyModel(1), status = ( a = 1, b = 0, c = 0), ) @@ -160,7 +160,7 @@ If you declare these functions without importing them first, or prefixing them w Forgetting to prefix the [`run!`](@ref) function definition gives the following error : ```julia -ERROR: MethodError: no method matching run!(::ModelList{@NamedTuple{…}, Status{…}}, ::TimeStepTable{Atmosphere{…}}) +ERROR: MethodError: no method matching run!(::ModelMapping{...}, ::TimeStepTable{Atmosphere{…}}) The function [`run!`](@ref) exists, but no method is defined for this combination of argument types. Closest candidates are: @@ -168,7 +168,7 @@ Closest candidates are: @ Main ~/path/to/file.jl:20 ``` -Forgetting to prefix the `inputs_`or `outputs_` functions for your model might not always generate an error, depending on whether the variables declared in this function are present in your ModelList or mapping's corresponding Status. +Forgetting to prefix the `inputs_`or `outputs_` functions for your model might not always generate an error, depending on whether the variables declared in this function are present in your mapping's corresponding Status. In cases where they do throw an error, you may get the following kind of output: ```julia @@ -234,7 +234,7 @@ The message 'got unsupported keyword argument "model"' can be misleading, as in A possible cause for this error is that a variable was declared instead of a symbol in a mapping for a multiscale model : ```julia -mapping = Dict("Scale" => +mapping = ModelMapping("Scale" => MultiScaleModel( model = ToyModel(), mapped_variables = [should_be_symbol => "Other_Scale"] # should_be_symbol is a variable, likely not found in the current module @@ -245,7 +245,7 @@ MultiScaleModel( Here's the correct version : ```julia -mapping = Dict("Scale" => +mapping = ModelMapping("Scale" => MultiScaleModel( model = ToyModel(), mapped_variables=[:should_be_symbol => "Other_Scale"] # should_be_symbol is now a symbol @@ -265,7 +265,7 @@ Here are a few examples when modifying the usual multiscale run! call in this wo mtg = Node(MultiScaleTreeGraph.NodeMTG("/", "Plant", 1, 1)) var1 = 15.0 - mapping = Dict( + mapping = ModelMapping( "Leaf" => ( Process1Model(1.0), Process2Model(), @@ -285,7 +285,7 @@ The exact signature is this : ```julia function run!( object::MultiScaleTreeGraph.Node, - mapping::Dict{String,T} where {T}, + mapping::ModelMapping, meteo=nothing, constants=PlantMeteo.Constants(), extra=nothing; @@ -350,7 +350,7 @@ However, the model provided in the examples, Process2Model is absent from the ma ```julia simple_mtg = Node(MultiScaleTreeGraph.NodeMTG("/", "Plant", 1, 1)) -mapping = Dict( +mapping = ModelMapping( "Leaf" => ( Process3Model(), Status(var5=15.0,) @@ -375,10 +375,10 @@ The fix is to add Process2Model() -or another model for the same process- to the One current problem with PlantSimEngine's API is that declaring a simulation's Status or Statuses differs between single- and multi-scale. -Returning to the example in [Implementing a model: forgetting to import or prefix functions](@ref), the `ModelList` status was declared like this: +Returning to the example in [Implementing a model: forgetting to import or prefix functions](@ref), the single-scale mapping status was declared like this: ```julia -model = ModelList( +model = ModelMapping( ToyToyModel(1), status = ( a = 1, b = 0, c = 0), ) @@ -447,12 +447,15 @@ An unintuitive error encountered in the past when defining a mapping : ```julia ERROR: ArgumentError: AbstractDict(kv): kv needs to be an iterator of 2-tuples or pairs ``` + may occur when forgetting the parenthesis after '=>' in a mapping declaration, and combining it with another parenthesis error. + ```julia -mapping = Dict( "Scale" => (ToyAssimGrowthModel(0.0, 0.0, 0.0), ToyCAllocationModel(), Status( TT_cu=Vector(cumsum(meteo_day.TT))), ), ) +mapping = ModelMapping( "Scale" => (ToyAssimGrowthModel(0.0, 0.0, 0.0), ToyCAllocationModel(), Status( TT_cu=Vector(cumsum(meteo_day.TT))), ), ) ``` -Other errors such as : +Other errors such as: + ```julia ERROR: MethodError: no method matching Dict(::Pair{String, ToyAssimGrowthModel{Float64}}, ::ToyCAllocationModel, ::Status{(:TT_cu,), Tuple{Base.RefValue{…}}}) The type `Dict` exists, but no method is defined for this combination of argument types when trying to construct it. @@ -460,6 +463,7 @@ The type `Dict` exists, but no method is defined for this combination of argumen Closest candidates are: Dict(::Pair{K, V}...) where {K, V} ``` + often indicate a likely syntax error somewhere in the mapping definition. ### Empty status vectors in multi-scale simulations @@ -486,7 +490,7 @@ function PlantSimEngine.outputs_(::ToyTt_CuModel) (TT_cu=-Inf,) end -mapping_multiscale = Dict( +mapping_multiscale = ModelMapping( "Scene" => ToyTt_CuModel(), "Plant" => ( MultiScaleModel( @@ -508,4 +512,4 @@ out_multiscale = run!(mtg_multiscale, mapping_multiscale, meteo_day) out_multiscale["Plant"][:LAI] ``` -In the above code, uncommenting the second line will add a "Plant" node to the MTG, and the simulation will then behave as intuitively expected. \ No newline at end of file +In the above code, uncommenting the second line will add a "Plant" node to the MTG, and the simulation will then behave as intuitively expected. diff --git a/docs/src/troubleshooting_and_testing/tips_and_workarounds.md b/docs/src/troubleshooting_and_testing/tips_and_workarounds.md index 07769d715..8f5f5bf3e 100644 --- a/docs/src/troubleshooting_and_testing/tips_and_workarounds.md +++ b/docs/src/troubleshooting_and_testing/tips_and_workarounds.md @@ -49,8 +49,8 @@ This change in design avoids model order ambiguity and also improves readability !!! note This section is a little more advanced and not recommended for beginners - -You may have noticed that sometimes a vector (1-dimensional array) variable is passed into the [`status`](@ref) component of a [`ModelList`](@ref) in documentation examples (An example here with cumulative thermal time : [Model switching](@ref)). + +You may have noticed that sometimes a vector (1-dimensional array) variable is passed into the [`status`](@ref) component of a [`ModelMapping`](@ref) in documentation examples (An example here with cumulative thermal time : [Model switching](@ref)). This is practical for simple simulations, or when quickly prototyping, to avoid having to write a model specifically for it. Whatever models make use of that variable are provided with one element corresponding to the current timestep every iteration. @@ -58,7 +58,7 @@ In multi-scale simulations, this feature is also supported, though not part of t It is more brittle, makes use of not-recommended Julia metaprogramming features (`eval()`), fiddles with global variables, might not work outside of a REPL environment and is not tested for more complex interactions, so it may interact badly with variables that are mapped to different scales or in bizarre dependency couplings. -Due to, uh, implementation quirks, the way to use this is as follows : +Due to, uh, implementation quirks, the way to use this is as follows: Call the function `replace_mapping_status_vectors_with_generated_models(mapping_with_vectors_in_status, timestep_model_organ_level, nsteps)`on your mapping. @@ -73,9 +73,7 @@ It will parse your mapping, generate custom models to store and feed the vector cumsum(meteo_day.TT) actually returns a CSV.SentinelArray.ChainedVectors{T, Vector{T}}, which is not a subtype of AbstractVector. Replacing it with Vector(cumsum(meteo_day.TT)) will provide an adequate type. -Here's an example usage, fixing the first attempt at [Converting a single-scale simulation to multi-scale -](@ref): - +Here's an example usage, fixing the first attempt at [Converting a single-scale simulation to multi-scale](@ref): ```julia using PlantSimEngine @@ -84,7 +82,7 @@ using PlantMeteo, CSV, DataFrames meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) # Direct translation of the single-scale simulation -mapping_pseudo_multiscale = Dict( +mapping_pseudo_multiscale = ModelMapping( "Plant" => ( ToyLAIModel(), Beer(0.5), diff --git a/docs/src/working_with_data/fitting.md b/docs/src/working_with_data/fitting.md index f946ae095..3554d9845 100644 --- a/docs/src/working_with_data/fitting.md +++ b/docs/src/working_with_data/fitting.md @@ -5,7 +5,7 @@ using PlantSimEngine, PlantMeteo, DataFrames, Statistics using PlantSimEngine.Examples meteo = Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65, Ri_PAR_f=300.0) -m = ModelList(Beer(0.6), status=(LAI=2.0,)) +m = ModelMapping(Beer(0.6), status=(LAI=2.0,)) run!(m, meteo) df = DataFrame(aPPFD=m[:aPPFD][1], LAI=m.status.LAI[1], Ri_PAR_f=meteo.Ri_PAR_f[1]) @@ -56,7 +56,7 @@ meteo = Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65, Ri_PAR_f=300.0) Computing the `PPFD` values from the `Ri_PAR_f` values using the `Beer` model (with `k=0.6`): ```@example usepkg -m = ModelList(Beer(0.6), status=(LAI=2.0,)) +m = ModelMapping(Beer(0.6), status=(LAI=2.0,)) run!(m, meteo) ``` diff --git a/docs/src/working_with_data/floating_point_accumulation_error.md b/docs/src/working_with_data/floating_point_accumulation_error.md index f17a482f9..6616a1d28 100644 --- a/docs/src/working_with_data/floating_point_accumulation_error.md +++ b/docs/src/working_with_data/floating_point_accumulation_error.md @@ -6,10 +6,10 @@ using PlantSimEngine.Examples using PlantMeteo, MultiScaleTreeGraph, CSV meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -models = ModelList( +models = ModelMapping( ToyLAIModel(), Beer(0.5), - ToyRUEGrowthModel(0.2), + ToyRUEGrowthModel(0.2); status=(TT_cu=cumsum(meteo_day.TT),), ) @@ -24,10 +24,10 @@ In the [Converting a single-scale simulation to multi-scale](@ref) page, a singl ```@example usepkg meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) -models_singlescale = ModelList( +models_singlescale = ModelMapping( ToyLAIModel(), Beer(0.5), - ToyRUEGrowthModel(0.2), + ToyRUEGrowthModel(0.2); status=(TT_cu=cumsum(meteo_day.TT),), ) @@ -54,7 +54,7 @@ function PlantSimEngine.outputs_(::ToyTt_CuModel) (TT_cu=0.0,) end -mapping_multiscale = Dict( +mapping_multiscale = ModelMapping( "Scene" => ToyTt_CuModel(), "Plant" => ( MultiScaleModel( @@ -77,7 +77,7 @@ outputs_multiscale = run!(mtg_multiscale, mapping_multiscale, meteo_day) ### Output comparison ```@setup usepkg -mapping_multiscale = Dict( +mapping_multiscale = ModelMapping( "Scene" => ToyTt_CuModel(), "Plant" => ( MultiScaleModel( @@ -157,4 +157,4 @@ Relating specifically to floating-point sums: Pairwise summation: [https://en.wikipedia.org/wiki/Pairwise_summation](https://en.wikipedia.org/wiki/Pairwise_summation) Kahan summation: [https://en.wikipedia.org/wiki/Kahan_summation_algorithm](https://en.wikipedia.org/wiki/Kahan_summation_algorithm) -Taming Floating-Point Sums: [https://orlp.net/blog/taming-float-sums/](https://orlp.net/blog/taming-float-sums/) \ No newline at end of file +Taming Floating-Point Sums: [https://orlp.net/blog/taming-float-sums/](https://orlp.net/blog/taming-float-sums/) diff --git a/docs/src/working_with_data/inputs.md b/docs/src/working_with_data/inputs.md index 4e5b4d004..dbc68f85c 100644 --- a/docs/src/working_with_data/inputs.md +++ b/docs/src/working_with_data/inputs.md @@ -1,6 +1,6 @@ # Input types -[`run!`](@ref) usually takes two inputs: a [`ModelList`](@ref) and data for the meteorology. The data for the meteorology is usually provided for one time step using an `Atmosphere`, or for several time-steps using a `TimeStepTable{Atmosphere}`. The [`ModelList`](@ref) can also be provided as a singleton, or as a vector or dictionary of. +[`run!`](@ref) usually takes two inputs: a [`ModelMapping`](@ref) and data for the meteorology. The data for the meteorology is usually provided for one time step using an `Atmosphere`, or for several time-steps using a `TimeStepTable{Atmosphere}`. The [`ModelMapping`](@ref) can also be provided as a singleton, or as a vector or dictionary of. [`run!`](@ref) knows how to handle these data formats via the [`PlantSimEngine.DataFormat`](@ref) trait (see [this blog post](https://www.juliabloggers.com/the-emergent-features-of-julialang-part-ii-traits/) to learn more about traits). For example, we tell PlantSimEngine that a `TimeStepTable` should be handled like a table by implementing the following trait: diff --git a/docs/src/working_with_data/reducing_dof.md b/docs/src/working_with_data/reducing_dof.md index 13fe72b20..e172fff9a 100644 --- a/docs/src/working_with_data/reducing_dof.md +++ b/docs/src/working_with_data/reducing_dof.md @@ -49,7 +49,7 @@ using PlantSimEngine, PlantMeteo using PlantSimEngine.Examples meteo = Atmosphere(T = 20.0, Wind = 1.0, P = 101.3, Rh = 0.65) -m = ModelList( +m = ModelMapping( Process1Model(2.0), Process2Model(), Process3Model(), @@ -68,7 +68,7 @@ status(m) Let's say that `m` is our complete model, and that we want to reduce the degrees of freedom by constraining the value of `var9` to a measurement, which was previously computed by `Process7Model`, a soft-dependency model. It is very easy to do this in PlantSimEngine: just remove the model from the model list and give the value of the measurement in the status: ```@example usepkg -m2 = ModelList( +m2 = ModelMapping( Process1Model(2.0), Process2Model(), Process3Model(), @@ -103,7 +103,7 @@ end Now we can create a new model list with the new model for `process7`: ```@example usepkg -m3 = ModelList( +m3 = ModelMapping( ForceProcess1Model(), Process2Model(), Process3Model(), diff --git a/docs/src/working_with_data/visualising_outputs.md b/docs/src/working_with_data/visualising_outputs.md index 54c7a940f..fb3279693 100644 --- a/docs/src/working_with_data/visualising_outputs.md +++ b/docs/src/working_with_data/visualising_outputs.md @@ -9,7 +9,7 @@ using PlantSimEngine.Examples meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) # Define the list of models for coupling: -model = ModelList( +model = ModelMapping( ToyLAIModel(), Beer(0.6), status=(TT_cu=cumsum(meteo_day[:, :TT]),), # Pass the cumulated degree-days as input to `ToyLAIModel`, this could also be done using another model @@ -39,7 +39,7 @@ using PlantSimEngine.Examples meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) # Define the list of models for coupling: -models = ModelList( +models = ModelMapping( ToyLAIModel(), Beer(0.6), status=(TT_cu=cumsum(meteo_day[:, :TT]),), # Pass the cumulated degree-days as input to `ToyLAIModel`, this could also be done using another model diff --git a/examples/Beer.jl b/examples/Beer.jl index f7d1893b0..c7069c536 100644 --- a/examples/Beer.jl +++ b/examples/Beer.jl @@ -35,7 +35,7 @@ of light extinction. # Arguments - `::Beer`: a Beer model, from the model list (*i.e.* m.light_interception) -- `models`: A `ModelList` struct holding the parameters for the model with +- `models`: A `ModelMapping` struct holding the parameters for the model with initialisations for `LAI` (m² m⁻²): the leaf area index. - `status`: the status of the model, usually the model list status (*i.e.* m.status) - `meteo`: meteorology structure, see [`Atmosphere`](https://palmstudio.github.io/PlantMeteo.jl/stable/#PlantMeteo.Atmosphere) @@ -45,7 +45,7 @@ initialisations for `LAI` (m² m⁻²): the leaf area index. # Examples ```julia -m = ModelList(Beer(0.5), status=(LAI=2.0,)) +m = ModelMapping(Beer(0.5), status=(LAI=2.0,)) meteo = Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65, Ri_PAR_q=300.0) @@ -95,7 +95,7 @@ using PlantSimEngine.Examples Create a model list with a Beer model, and fit it to the data: ```julia -m = ModelList(Beer(0.6), status=(LAI=2.0,)) +m = ModelMapping(Beer(0.6), status=(LAI=2.0,)) meteo = Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65, Ri_PAR_f=300.0) run!(m, meteo) df = DataFrame(aPPFD=m[:aPPFD][1], LAI=m.status.LAI[1], Ri_PAR_f=meteo.Ri_PAR_f[1]) diff --git a/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation1.jl b/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation1.jl index bd9ff95f5..298b83dac 100644 --- a/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation1.jl +++ b/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation1.jl @@ -96,7 +96,7 @@ end PlantSimEngine.ObjectDependencyTrait(::Type{<:ToyLeafCarbonCaptureModel}) = PlantSimEngine.IsObjectIndependent() PlantSimEngine.TimeStepDependencyTrait(::Type{<:ToyLeafCarbonCaptureModel}) = PlantSimEngine.IsTimeStepIndependent() -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( diff --git a/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation2.jl b/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation2.jl index 393ec9c2c..9f156bca1 100644 --- a/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation2.jl +++ b/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation2.jl @@ -176,7 +176,7 @@ end PlantSimEngine.ObjectDependencyTrait(::Type{<:ToyLeafCarbonCaptureModel}) = PlantSimEngine.IsObjectIndependent() PlantSimEngine.TimeStepDependencyTrait(::Type{<:ToyLeafCarbonCaptureModel}) = PlantSimEngine.IsTimeStepIndependent() -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( diff --git a/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation3.jl b/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation3.jl index eb5553967..cda458e84 100644 --- a/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation3.jl +++ b/examples/ToyMultiScalePlantTutorial/ToyPlantSimulation3.jl @@ -193,7 +193,7 @@ function PlantSimEngine.run!(::ToyLeafCarbonCaptureModel, models, status, meteo, end -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( diff --git a/examples/ToySingleToMultiScale.jl b/examples/ToySingleToMultiScale.jl index a65a90573..2f4ee787e 100644 --- a/examples/ToySingleToMultiScale.jl +++ b/examples/ToySingleToMultiScale.jl @@ -17,7 +17,7 @@ meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), ### Single-scale simulation ############################## -models_singlescale = ModelList( +models_singlescale = ModelMapping( ToyLAIModel(), Beer(0.5), ToyRUEGrowthModel(0.2), @@ -29,12 +29,12 @@ outputs_singlescale = run!(models_singlescale, meteo_day) ############################## #### Direct translation of the single-scale simulation ############################## -mapping_pseudo_multiscale = Dict( -"Plant" => ( - ToyLAIModel(), - Beer(0.5), - ToyRUEGrowthModel(0.2), - Status(TT_cu=cumsum(meteo_day.TT),) +mapping_pseudo_multiscale = ModelMapping( + "Plant" => ( + ToyLAIModel(), + Beer(0.5), + ToyRUEGrowthModel(0.2), + Status(TT_cu=cumsum(meteo_day.TT),) ), ) @@ -69,7 +69,7 @@ end #### Actual multiscale version of the single-scale simulation ############################## -mapping_multiscale = Dict( +mapping_multiscale = ModelMapping( "Scene" => ( ToyTt_CuModel(), Status(TT_cu=0.0), @@ -87,9 +87,9 @@ mapping_multiscale = Dict( ) # We now need two nodes for our MTG -mtg_multiscale = MultiScaleTreeGraph.Node(MultiScaleTreeGraph.NodeMTG("/", "Scene", 1, 0)) - plant = MultiScaleTreeGraph.Node(mtg_multiscale, MultiScaleTreeGraph.NodeMTG("+", "Plant", 1, 1)) - outputs_multiscale = run!(mtg_multiscale, mapping_multiscale, meteo_day) +mtg_multiscale = MultiScaleTreeGraph.Node(MultiScaleTreeGraph.NodeMTG("/", "Scene", 1, 0)) +plant = MultiScaleTreeGraph.Node(mtg_multiscale, MultiScaleTreeGraph.NodeMTG("+", "Plant", 1, 1)) +outputs_multiscale = run!(mtg_multiscale, mapping_multiscale, meteo_day) ############################## #### Output comparison @@ -116,5 +116,5 @@ is_approx_equal_2 = length(unique(computed_TT_cu_multiscale .≈ outputs_singles is_perfectly_equal = length(unique(computed_TT_cu_multiscale .== outputs_singlescale.TT_cu)) == 1 -(computed_TT_cu_multiscale .== outputs_singlescale.TT_cu)[104] -(computed_TT_cu_multiscale .== outputs_singlescale.TT_cu)[105] +(computed_TT_cu_multiscale.==outputs_singlescale.TT_cu)[104] +(computed_TT_cu_multiscale.==outputs_singlescale.TT_cu)[105] diff --git a/examples/benchmark.jl b/examples/benchmark.jl index 0ecfdbfc6..a372ad467 100644 --- a/examples/benchmark.jl +++ b/examples/benchmark.jl @@ -5,7 +5,7 @@ using PlantSimEngine, PlantMeteo, DataFrames, CSV, Dates, Statistics # using PlantSimEngine.Examples meteo_day = read_weather(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), duration=Day) -models = ModelList( +models = ModelMapping( ToyLAIModel(), status=(TT_cu=cumsum(meteo_day.TT),), ) @@ -20,7 +20,7 @@ time_run_seq = @benchmark run!($models, $meteo_day, executor=$(SequentialEx())) median_time_seq_ns = median(time_run_seq.times) / nrow(meteo_day) # Coupled model: -models_coupled = ModelList( +models_coupled = ModelMapping( ToyLAIModel(), Beer(0.5), status=(TT_cu=cumsum(meteo_day.TT),), diff --git a/src/PlantSimEngine.jl b/src/PlantSimEngine.jl index 62e6fee46..1f9ec5619 100644 --- a/src/PlantSimEngine.jl +++ b/src/PlantSimEngine.jl @@ -51,9 +51,10 @@ include("component_models/TimeStepTable.jl") include("dependencies/dependency_graph.jl") # List of models: -include("component_models/ModelList.jl") +include("component_models/ModelList.jl") # deprecated, to be removed in favor of ModelMapping include("mtg/MultiScaleModel.jl") include("mtg/ModelSpec.jl") +include("mtg/mapping/mapping.jl") # Getters / setters for status: include("component_models/get_status.jl") @@ -73,7 +74,6 @@ include("dependencies/get_model_in_dependency_graph.jl") # MTG compatibility: include("mtg/GraphSimulation.jl") include("mtg/mapping/getters.jl") -include("mtg/mapping/mapping.jl") include("mtg/mapping/compute_mapping.jl") include("mtg/mapping/reverse_mapping.jl") include("mtg/model_spec_inference.jl") @@ -124,7 +124,7 @@ export AbstractTimeReducer, MeanWeighted, MeanReducer, SumReducer, MinReducer, M export OutputCache, HoldLastCache, InterpolateCache, IntegrateCache, AggregateCache export TemporalState export OutputRequest, collect_outputs -export ModelList, MultiScaleModel, ModelSpec, TimeStepModel, InputBindings, MeteoBindings, MeteoWindow, OutputRouting, ScopeModel +export ModelList, MultiScaleModel, ModelMapping, ModelSpec, TimeStepModel, InputBindings, MeteoBindings, MeteoWindow, OutputRouting, ScopeModel export resolved_model_specs, explain_model_specs export RMSE, NRMSE, EF, dr export Status, TimeStepTable, status diff --git a/src/checks/dimensions.jl b/src/checks/dimensions.jl index b8bd22e62..46a29959e 100644 --- a/src/checks/dimensions.jl +++ b/src/checks/dimensions.jl @@ -16,7 +16,7 @@ using PlantSimEngine.Examples w = Atmosphere(T = 20.0, Rh = 0.5, Wind = 1.0) # Creating a dummy component: -models = ModelList( +models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -47,6 +47,10 @@ function check_dimensions(component::T, w) where {T<:ModelList} check_dimensions(status(component), w) end +function check_dimensions(component::ModelMapping{SingleScale}, w) + check_dimensions(status(component), w) +end + # for several components as an array function check_dimensions(component::T, weather) where {T<:AbstractArray{<:ModelList}} for i in component @@ -82,7 +86,7 @@ end function check_dimensions(::SingletonAlike, st::Status, weather) for (var, value) in zip(keys(st), st) - if length(value) > 1 + if length(value) > 1 throw(DimensionMismatch("Component status has a vector variable : $(var) implying multiple timesteps but weather data only provides a single timestep.")) end end @@ -110,4 +114,4 @@ end function get_nsteps(::TableAlike, t) DataAPI.nrow(t) -end \ No newline at end of file +end diff --git a/src/component_models/ModelList.jl b/src/component_models/ModelList.jl index a8a21938f..ae88b37e0 100644 --- a/src/component_models/ModelList.jl +++ b/src/component_models/ModelList.jl @@ -106,7 +106,7 @@ julia> outputs_sim[:var6] If we want to use special types for the variables, we can use the `type_promotion` argument: ```jldoctest 1 -julia> models = ModelList(process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), status=(var1=15.0, var2=0.3), type_promotion = Dict(Float64 => Float32)); +julia> models = ModelList(process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), status=(var1=15.0, var2=0.3), type_promotion = ModelMapping(Float64 => Float32)); ``` We used `type_promotion` to force the status into Float32: @@ -163,7 +163,6 @@ function ModelList( variables_check::Bool=true, kwargs... ) - # Get all the variables needed by the models and their default values: if length(args) > 0 args = parse_models(args) diff --git a/src/component_models/get_status.jl b/src/component_models/get_status.jl index 5a17e4823..55327614a 100644 --- a/src/component_models/get_status.jl +++ b/src/component_models/get_status.jl @@ -16,7 +16,7 @@ using PlantSimEngine using PlantSimEngine.Examples; # Create a ModelList -models = ModelList( +models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -44,6 +44,10 @@ function status(m) m.status end +status(m::ModelMapping{SingleScale}) = status(m.data) +status(m::ModelMapping{SingleScale}, key::Symbol) = status(m.data, key) +status(m::ModelMapping{SingleScale}, key::T) where {T<:Integer} = status(m.data, key) + function status(m::T) where {T<:AbstractArray{M} where {M}} [status(i) for i in m] end diff --git a/src/dataframe.jl b/src/dataframe.jl index a47e67e77..e36c153f7 100644 --- a/src/dataframe.jl +++ b/src/dataframe.jl @@ -1,9 +1,8 @@ """ - DataFrame(components <: AbstractArray{<:ModelList}) - DataFrame(components <: AbstractDict{N,<:ModelList}) + DataFrame(components <: AbstractArray{<:ModelMapping}) + DataFrame(components <: AbstractDict{N,<:ModelMapping}) -Fetch the data from a [`ModelList`](@ref) (or an Array/Dict of) status into -a DataFrame. +Fetch the data from a [`ModelMapping`](@ref) (or an Array/Dict of) status into a DataFrame. # Examples @@ -11,8 +10,8 @@ a DataFrame. using PlantSimEngine using DataFrames -# Creating a ModelList -models = ModelList( +# Creating a ModelMapping +models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -22,14 +21,14 @@ models = ModelList( # Converting to a DataFrame df = DataFrame(models) -# Converting to a Dict of ModelLists -models = Dict( - "Leaf" => ModelList( +# Converting to a Dict of ModelMappings +models = ModelMapping( + "Leaf" => ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model() ), - "InterNode" => ModelList( + "InterNode" => ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model() @@ -40,7 +39,7 @@ models = Dict( df = DataFrame(models) ``` """ -function DataFrames.DataFrame(components::T) where {T<:AbstractArray{<:ModelList}} +function DataFrames.DataFrame(components::T) where {T<:AbstractArray{<:ModelMapping}} df = DataFrame[] for (k, v) in enumerate(components) df_c = DataFrames.DataFrame(v) @@ -50,7 +49,7 @@ function DataFrames.DataFrame(components::T) where {T<:AbstractArray{<:ModelList reduce(vcat, df) end -function DataFrames.DataFrame(components::T) where {T<:AbstractDict{N,<:ModelList} where {N}} +function DataFrames.DataFrame(components::T) where {T<:AbstractDict{N,<:ModelMapping} where {N}} df = DataFrames.DataFrame[] for (k, v) in components df_c = DataFrames.DataFrame(v) @@ -61,10 +60,10 @@ function DataFrames.DataFrame(components::T) where {T<:AbstractDict{N,<:ModelLis end """ - DataFrame(components::ModelList{T,S}) where {T,S<:Status} + DataFrame(components::ModelMapping{T,S}) where {T,S<:Status} -Implementation of `DataFrame` for a `ModelList` model with one time step. +Implementation of `DataFrame` for a `ModelMapping` model with one time step. """ -function DataFrames.DataFrame(components::ModelList{T,S}) where {T,S<:Status} +function DataFrames.DataFrame(components::ModelMapping{T}) where {T} DataFrames.DataFrame([NamedTuple(status(components)[1])]) end diff --git a/src/dependencies/dependencies.jl b/src/dependencies/dependencies.jl index 94750274c..612e16db5 100644 --- a/src/dependencies/dependencies.jl +++ b/src/dependencies/dependencies.jl @@ -1,11 +1,11 @@ dep(::T, nsteps=1) where {T<:AbstractModel} = NamedTuple() """ - dep(m::ModelList) - dep(mapping::Dict{String,T}; verbose=true) - dep!(m::ModelList, nsteps=1) + dep(mapping::ModelMapping; verbose=true) + dep(mapping::AbstractDict{String,T}; verbose=true) + dep!(m::ModelMapping, nsteps=1) -Get the model dependency graph given a ModelList or a multiscale model mapping. If one graph is returned, +Get the model dependency graph given a ModelMapping or a multiscale model mapping. If one graph is returned, then all models are coupled. If several graphs are returned, then only the models inside each graph are coupled, and the models in different graphs are not coupled. `nsteps` is the number of steps the dependency graph will be used over. It is used to determine @@ -31,7 +31,7 @@ Note that in the 5th case, we still need to check if a variable is needed from a used as a child of the process at the other scale. Note there can be several levels of hard dependency graph, so this is done recursively. How do we do all that? We identify the hard dependencies first. Then we link the inputs/outputs of the hard dependencies roots -to other scales if needed. Then we transform all these nodes into soft dependencies, that we put into a Dict of Scale => Dict(process => SoftDependencyNode). +to other scales if needed. Then we transform all these nodes into soft dependencies, that we put into a Dict of Scale => ModelMapping(process => SoftDependencyNode). Then we traverse all these and we set nodes that need outputs from other nodes as inputs as children/parents. If a node has no dependency, it is set as a root node and pushed into a new Dict (independant_process_root). This Dict is the returned dependency graph. And it presents root nodes as independent starting points for the sub-graphs, which are the models that are coupled together. We can then traverse each of @@ -39,7 +39,7 @@ these graphs independently to r # Notes -The difference between `dep(m::ModelList)` and `dep!(m::ModelList, nsteps)` is that the first one returns the dependency graph found in the model list, while the +The difference between `dep(m::ModelMapping)` and `dep!(m::ModelMapping, nsteps)` is that the first one returns the dependency graph found in the model list, while the second one returns the dependency graph with the specified number of steps, modifying the simulation IDs of each node in the graph (`simulation_id=fill(0, nsteps)`). # Examples @@ -50,7 +50,7 @@ using PlantSimEngine # Including example processes and models: using PlantSimEngine.Examples; -models = ModelList( +models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -100,15 +100,15 @@ function dep(m::NamedTuple, nsteps=1; verbose::Bool=true) dep(nsteps; verbose=verbose, m...) end -function dep(mapping::Dict{String,T}; verbose::Bool=true) where {T} +function dep(mapping::AbstractDict{String,T}; verbose::Bool=true) where {T} # First step, get the hard-dependency graph and create SoftDependencyNodes for each hard-dependency root. In other word, we want # only the nodes that are not hard-dependency of other nodes. These nodes are taken as roots for the soft-dependency graph because they # are independant. soft_dep_graphs_roots, hard_dep_dict = hard_dependencies(mapping; verbose=verbose) - + mapped_vars = mapped_variables(mapping, soft_dep_graphs_roots, verbose=false) reverse_multiscale_mapping = reverse_mapping(mapped_vars, all=false) - + # Second step, compute the soft-dependency graph between SoftDependencyNodes computed in the first step. To do so, we search the # inputs of each process into the outputs of the other processes, at the same scale, but also between scales. Then we keep only the # nodes that have no soft-dependencies, and we set them as root nodes of the soft-dependency graph. The other nodes are set as children diff --git a/src/dependencies/hard_dependencies.jl b/src/dependencies/hard_dependencies.jl index 0d1fa5b3c..dc70d2d82 100644 --- a/src/dependencies/hard_dependencies.jl +++ b/src/dependencies/hard_dependencies.jl @@ -1,6 +1,7 @@ """ hard_dependencies(models; verbose::Bool=true) - hard_dependencies(mapping::Dict{String,T}; verbose::Bool=true) + hard_dependencies(mapping::ModelMapping; verbose::Bool=true) + hard_dependencies(mapping::AbstractDict{String,T}; verbose::Bool=true) Compute the hard dependencies between models. """ @@ -112,7 +113,7 @@ end # When we use a mapping (multiscale), we return the set of soft-dependencies (we put the hard-dependencies as their children): -function hard_dependencies(mapping::Dict{String,T}; verbose::Bool=true) where {T} +function hard_dependencies(mapping::AbstractDict{String,T}; verbose::Bool=true) where {T} full_vars_mapping = Dict(first(mod) => Dict(get_mapped_variables(last(mod))) for mod in mapping) soft_dep_graphs = Dict{String,Any}() not_found = Dict{Symbol,DataType}() @@ -256,4 +257,4 @@ function hard_dependencies(mapping::Dict{String,T}; verbose::Bool=true) where {T end return (DependencyGraph(soft_dep_graphs, not_found), hard_dependency_dict) -end \ No newline at end of file +end diff --git a/src/doc_templates/mtg-related.jl b/src/doc_templates/mtg-related.jl index c5c3666dc..3cdc90c7e 100644 --- a/src/doc_templates/mtg-related.jl +++ b/src/doc_templates/mtg-related.jl @@ -30,7 +30,7 @@ leaf2 = Node(internode2, MultiScaleTreeGraph.NodeMTG("+", "Leaf", 1, 2)); const MAPPING_EXAMPLE = """ ```@example -mapping = Dict( \ +mapping = ModelMapping( \ "Plant" => ( \ MultiScaleModel( \ model=ToyCAllocationModel(), \ diff --git a/src/mtg/GraphSimulation.jl b/src/mtg/GraphSimulation.jl index 2b7cbcab2..9297f276c 100644 --- a/src/mtg/GraphSimulation.jl +++ b/src/mtg/GraphSimulation.jl @@ -34,7 +34,8 @@ struct GraphSimulation{T,S,U,O,V,TS,MS} end function GraphSimulation(graph, mapping; nsteps=1, outputs=nothing, type_promotion=nothing, check=true, verbose=false) - GraphSimulation(init_simulation(graph, mapping; nsteps=nsteps, outputs=outputs, type_promotion=type_promotion, check=check, verbose=verbose)...) + mapping_checked = mapping isa ModelMapping ? mapping : ModelMapping(mapping) + GraphSimulation(init_simulation(graph, mapping_checked; nsteps=nsteps, outputs=outputs, type_promotion=type_promotion, check=check, verbose=verbose)...) end dep(g::GraphSimulation) = g.dependency_graph diff --git a/src/mtg/initialisation.jl b/src/mtg/initialisation.jl index 82deaab3d..e9661bb8d 100644 --- a/src/mtg/initialisation.jl +++ b/src/mtg/initialisation.jl @@ -278,7 +278,7 @@ Initialise the simulation. Returns: # Arguments - `mtg`: the MTG -- `mapping::Dict{String,Any}`: a dictionary of model mapping +- `mapping::ModelMapping` (or dictionary-like mapping): associates scales to models/status. - `nsteps`: the number of steps of the simulation - `outputs`: the dynamic outputs needed for the simulation - `type_promotion`: the type promotion to use for the variables diff --git a/src/mtg/mapping/compute_mapping.jl b/src/mtg/mapping/compute_mapping.jl index 21d640081..d262f685c 100644 --- a/src/mtg/mapping/compute_mapping.jl +++ b/src/mtg/mapping/compute_mapping.jl @@ -5,7 +5,7 @@ Get the variables for each organ type from a dependency graph, with `MappedVar`s # Arguments -- `mapping::Dict{String,T}`: the mapping between models and scales. +- `mapping::ModelMapping` (or dictionary-like mapping): the mapping between models and scales. - `dependency_graph::DependencyGraph`: the first-order dependency graph where each model in the mapping is assigned a node. However, models that are identified as hard-dependencies are not given individual nodes. Instead, they are nested as child nodes under other models. @@ -42,7 +42,7 @@ Get the variables for each organ type from a dependency graph, without the varia # Arguments -- `mapping::Dict{String,T}`: the mapping between models and scales. +- `mapping::ModelMapping` (or dictionary-like mapping): the mapping between models and scales. - `dependency_graph::DependencyGraph`: the first-order dependency graph where each model in the mapping is assigned a node. However, models that are identified as hard-dependencies are not given individual nodes. Instead, they are nested as child nodes under other models. @@ -353,4 +353,4 @@ function convert_reference_values!(mapped_vars::Dict{String,Dict{Symbol,Any}}) end end return mapped_vars -end \ No newline at end of file +end diff --git a/src/mtg/mapping/getters.jl b/src/mtg/mapping/getters.jl index b6a465473..82125af1b 100644 --- a/src/mtg/mapping/getters.jl +++ b/src/mtg/mapping/getters.jl @@ -5,7 +5,7 @@ Get the models of a dictionary of model mapping. # Arguments -- `m::Dict{String,Any}`: a dictionary of model mapping +- `m`: a scale mapping entry (for example one value from a [`ModelMapping`](@ref)) Returns a vector of models @@ -92,7 +92,7 @@ Get the status of a dictionary of model mapping. # Arguments -- `m::Dict{String,Any}`: a dictionary of model mapping +- `m`: a scale mapping entry (for example one value from a [`ModelMapping`](@ref)) Returns a [`Status`](@ref) or `nothing`. @@ -114,7 +114,7 @@ Get the mapping of a dictionary of model mapping. # Arguments -- `m::Dict{String,Any}`: a dictionary of model mapping +- `m`: a scale mapping entry (for example one value from a [`ModelMapping`](@ref)) Returns a vector of pairs of symbols and strings or vectors of strings diff --git a/src/mtg/mapping/mapping.jl b/src/mtg/mapping/mapping.jl index 5f81dec85..536b8cc3a 100755 --- a/src/mtg/mapping/mapping.jl +++ b/src/mtg/mapping/mapping.jl @@ -87,4 +87,390 @@ end mapped_default(m::MappedVar) = m.source_default mapped_default(m::MappedVar{O,V1,V2,T}, organ) where {O<:MultiNodeMapping,V1,V2<:Vector{Symbol},T} = m.source_default[findfirst(o -> o == organ, mapped_organ(m))] -mapped_default(m) = m # For any variable that is not a MappedVar, we return it as is \ No newline at end of file +mapped_default(m) = m # For any variable that is not a MappedVar, we return it as is + +# This defines the type of mapping setup: either single or multiscale. Used to dispatch methods for e.g. `dep` or `to_initialize`. +abstract type AbstractScaleSetup end + +struct MultiScale <: AbstractScaleSetup end +struct SingleScale <: AbstractScaleSetup end + +""" + ModelMapping(mapping; check=true) + +Validated mapping between MTG scales and model definitions. + +Each scale entry may be provided as: +- a single model, +- a tuple of models with an optional [`Status`](@ref), + +At construction time, the mapping is normalized and checked to fail early on common +configuration errors: +- each scale must define at least one model, +- at most one `Status` is allowed per scale, +- mapped scales must exist in the mapping, +- mapped source variables must exist on the source scale (as a model output or status variable), +- duplicate process declarations at a given scale are rejected. + +# Notes + +The type behaves like a read-only dictionary keyed by scale name (`String`). +Use `Dict(mapping)` to recover a plain dictionary. +""" +struct ModelMapping{S<:AbstractScaleSetup,D} <: AbstractDict{String,Tuple} where {D<:Union{Dict{String,Tuple},ModelList}} + data::D +end + +ModelMapping{S}(data) where {S<:AbstractScaleSetup} = ModelMapping{S,typeof(data)}(data) + +""" + model_rate(model::AbstractModel) + +Optional model hook used by [`ModelMapping`](@ref) to check rate compatibility. + +By default it returns `nothing` (no explicit rate contract). Package users can provide +model-specific methods that return a comparable value (for example `Dates.Period`), and +`ModelMapping` will reject incompatible mapped couplings. +""" +model_rate(::AbstractModel) = nothing +model_rate(model::MultiScaleModel) = model_rate(model_(model)) + +Base.length(mapping::ModelMapping{MultiScale}) = length(mapping.data) +Base.length(::ModelMapping{SingleScale}) = 1 +Base.iterate(mapping::ModelMapping{MultiScale}, state...) = iterate(mapping.data, state...) +# Base.iterate(mapping::ModelMapping{SingleScale}, state...) = iterate(mapping.data.models, state...) +Base.show(io::IO, mapping::ModelMapping) = print(io, "ModelMapping with scales: ", join(keys(mapping), ", ")) +# Base.show(io::IO, mapping::ModelMapping{SingleScale}) = print(io, "Single Scale ModelMapping:\n", mapping.data.models) +Base.show(io::IO, mapping::ModelMapping{SingleScale}) = print(io, "Single Scale ModelMapping") + +function Base.show(io::IO, m::MIME"text/plain", t::ModelMapping{SingleScale}) + print(io, "Single Scale ModelMapping:\n") + show(io, m, t.data) +end + +function Base.show(io::IO, m::MIME"text/plain", t::ModelMapping) + print(io, "ModelMapping with scales: ", join(keys(t), ", ")) +end + +Base.keys(mapping::ModelMapping) = keys(mapping.data) +Base.values(mapping::ModelMapping) = values(mapping.data) +Base.pairs(mapping::ModelMapping) = pairs(mapping.data) +Base.keys(::ModelMapping{SingleScale}) = ("Default",) +Base.values(mapping::ModelMapping{SingleScale}) = ((values(mapping.data.models)..., status(mapping.data)),) +Base.pairs(mapping::ModelMapping{SingleScale}) = ("Default" => (values(mapping.data.models)..., status(mapping.data)),) +Base.getindex(mapping::ModelMapping, key::String) = mapping.data[key] +Base.getindex(mapping::ModelMapping, key::AbstractString) = mapping.data[String(key)] +function Base.getindex(mapping::ModelMapping{SingleScale}, key::String) + key == "Default" || throw(KeyError(key)) + return (values(mapping.data.models)..., status(mapping.data)) +end +Base.getindex(mapping::ModelMapping{SingleScale}, key::AbstractString) = getindex(mapping, String(key)) +Base.getindex(mapping::ModelMapping{SingleScale}, key::Symbol) = getindex(mapping.data, key) +Base.getindex(mapping::ModelMapping{SingleScale}, key::Integer) = getindex(mapping.data, key) +Base.haskey(mapping::ModelMapping, key::String) = haskey(mapping.data, key) +Base.haskey(mapping::ModelMapping, key::AbstractString) = haskey(mapping.data, String(key)) +Base.eltype(::Type{ModelMapping}) = Pair{String,Tuple} +Base.copy(mapping::ModelMapping{MultiScale}) = ModelMapping(copy(mapping.data); check=false) +Base.copy(mapping::ModelMapping{SingleScale}) = ModelMapping{SingleScale,ModelList}(copy(mapping.data)) +Base.copy(mapping::ModelMapping{SingleScale}, status) = ModelMapping{SingleScale,ModelList}(copy(mapping.data, status)) +Base.Dict(mapping::ModelMapping) = copy(mapping.data) +Base.:(==)(left::ModelMapping{SingleScale}, right::ModelMapping{SingleScale}) = left.data == right.data + +function Base.getproperty(mapping::ModelMapping{SingleScale}, name::Symbol) + name === :data && return getfield(mapping, :data) + return getproperty(getfield(mapping, :data), name) +end + +function ModelMapping{MultiScale}(mapping::T; check::Bool=true) where {T<:AbstractDict} + normalized = _normalize_multiscale_mapping(mapping) + check && _check_multiscale_mapping!(normalized) + ModelMapping{MultiScale,Dict{String,Tuple}}(normalized) +end + +ModelMapping(mapping::AbstractDict; check::Bool=true) = ModelMapping{MultiScale}(mapping; check=check) + +ModelMapping(mapping::ModelMapping; check::Bool=true) = check ? ModelMapping(mapping.data; check=true) : mapping + +""" + ModelMapping(scale_mapping_pairs...; check=true) + ModelMapping(models...; scale="Default", status=nothing, check=true, processes...) + +Convenience constructors for [`ModelMapping`](@ref): + +- pass `scale => models` pairs directly (dict-like syntax), +- or pass models/processes directly for a single scale (old `ModelList` syntax). +""" +function ModelMapping( + args...; + scale::AbstractString="Default", + status=nothing, + check::Bool=true, + processes... +) + isempty(args) && isempty(processes) && error( + "No mapping or model was provided. Use `ModelMapping(\"Scale\" => models)` or pass models directly." + ) + + # Backwards compatibility: allow dict-like construction for type promotion maps, + # e.g. `ModelMapping(Float64 => Float32)`. + if !isempty(args) && all(arg -> arg isa Pair && !(first(arg) isa Union{AbstractString,Symbol}), args) + return Dict(args) + end + + if _all_scale_pairs(args) + isempty(processes) || error( + "Cannot mix scale-level pairs with process keyword arguments. ", + "Use either `\"Scale\" => models` pairs, or single-scale process/model arguments." + ) + isnothing(status) || error( + "`status` cannot be used with scale-level pair syntax. ", + "Provide statuses inside each scale mapping instead." + ) + raw_mapping = Dict{String,Any}(String(first(pair)) => last(pair) for pair in args) + return ModelMapping{MultiScale}(raw_mapping; check=check) + end + + _contains_scale_like_pair(args) && error( + "Invalid argument mix: scale-level pairs must not be mixed with model arguments." + ) + + flat_args = Any[] + for arg in args + if arg isa Pair && first(arg) isa Symbol + push!(flat_args, last(arg)) + elseif arg isa NamedTuple + append!(flat_args, values(arg)) + elseif arg isa Tuple + append!(flat_args, arg) + else + push!(flat_args, arg) + end + end + + return ModelMapping{SingleScale,ModelList}(ModelList(flat_args...; status=status, type_promotion=nothing, variables_check=check, processes...)) + + #TODO: Use the following when we merge the ModelList and ModelMapping paths (create a fake scale): + single_scale_models = _single_scale_mapping_entries(args, processes, status) + # return ModelMapping{SingleScale}(Dict(String(scale) => single_scale_models), check=check) +end + +# Canonical API dispatches for model mappings. +dep(mapping::ModelMapping{SingleScale}; verbose::Bool=true) = dep(mapping.data) +dep(mapping::ModelMapping{MultiScale}; verbose::Bool=true) = dep(mapping.data; verbose=verbose) +hard_dependencies(mapping::ModelMapping{SingleScale}; verbose::Bool=true) = hard_dependencies(mapping.data) +hard_dependencies(mapping::ModelMapping{MultiScale}; verbose::Bool=true) = hard_dependencies(mapping.data; verbose=verbose) +inputs(mapping::ModelMapping) = inputs(mapping.data) +outputs(mapping::ModelMapping) = outputs(mapping.data) +variables(mapping::ModelMapping) = variables(mapping.data) +to_initialize(mapping::ModelMapping, graph=nothing) = to_initialize(mapping.data, graph) +reverse_mapping(mapping::ModelMapping; all=true) = reverse_mapping(mapping.data; all=all) +init_variables(mapping::ModelMapping{SingleScale}; verbose=true) = init_variables(mapping.data; verbose=verbose) +to_initialize(mapping::ModelMapping{SingleScale}) = to_initialize(mapping.data) +to_initialize(mapping::ModelMapping{SingleScale}, graph) = to_initialize(mapping) +pre_allocate_outputs(mapping::ModelMapping{SingleScale}, outs, nsteps; type_promotion=nothing, check=true) = + pre_allocate_outputs(mapping.data, outs, nsteps; type_promotion=type_promotion, check=check) + +function _all_scale_pairs(args) + !isempty(args) && all(arg -> arg isa Pair && first(arg) isa Union{AbstractString,Symbol}, args) +end + +function _contains_scale_like_pair(args) + any(arg -> arg isa Pair && first(arg) isa Union{AbstractString,Symbol}, args) +end + +function _single_scale_mapping_entries(args, processes, status) + models = Any[] + + for arg in args + if arg isa Pair && first(arg) isa Symbol + push!(models, last(arg)) + elseif arg isa NamedTuple + append!(models, values(arg)) + elseif arg isa Tuple + append!(models, arg) + else + push!(models, arg) + end + end + + append!(models, values(processes)) + + if !isnothing(status) + status_entry = status isa Status ? status : Status(status) + push!(models, status_entry) + end + + return tuple(models...) +end + +function _normalize_multiscale_mapping(mapping::AbstractDict) + isempty(mapping) && error("ModelMapping cannot be empty. Provide at least one scale with models.") + normalized = Dict{String,Tuple}() + for (scale, scale_mapping) in pairs(mapping) + scale_name = String(scale) + normalized[scale_name] = _normalize_scale_mapping(scale_name, scale_mapping) + end + return normalized +end + +function _normalize_scale_mapping(scale::String, scale_mapping::ModelList) + return _normalize_scale_mapping(scale, (values(scale_mapping.models)..., status(scale_mapping))) +end + +function _normalize_scale_mapping(scale::String, scale_mapping::ModelMapping{SingleScale}) + return _normalize_scale_mapping(scale, scale_mapping.data) +end + +function _normalize_scale_mapping(scale::String, scale_mapping::Union{AbstractModel,MultiScaleModel,ModelSpec}) + return (scale_mapping,) +end + +function _normalize_scale_mapping(scale::String, scale_mapping::Tuple) + normalized_items = Any[] + for item in scale_mapping + if item isa ModelList + append!(normalized_items, values(item.models)) + push!(normalized_items, status(item)) + elseif item isa Union{AbstractModel,MultiScaleModel,ModelSpec,Status} + push!(normalized_items, item) + else + error( + "Invalid mapping entry at scale `$scale`: expected models/ModelSpec, Status, or ModelList, got $(typeof(item))." + ) + end + end + return tuple(normalized_items...) +end + +function _normalize_scale_mapping(scale::String, scale_mapping) + error( + "Invalid mapping entry at scale `$scale`: expected a model/ModelSpec, tuple of models/Status, or ModelList, got $(typeof(scale_mapping))." + ) +end + +function _check_multiscale_mapping!(mapping::Dict{String,Tuple}) + _check_scales_have_models!(mapping) + _check_scale_process_uniqueness!(mapping) + _check_mapped_sources_exist!(mapping) + return mapping +end + +function _check_scales_have_models!(mapping::Dict{String,Tuple}) + for (scale, scale_mapping) in mapping + n_status = count(item -> item isa Status, scale_mapping) + n_status > 1 && error("Scale `$scale` defines $n_status statuses. Only one Status is allowed per scale.") + + models = get_models(scale_mapping) + isempty(models) && error( + "Scale `$scale` defines no model. Add at least one model, or remove this scale from the mapping." + ) + end +end + +function _check_scale_process_uniqueness!(mapping::Dict{String,Tuple}) + for (scale, scale_mapping) in mapping + process_names = [_process_name_for_mapping_check(model) for model in get_models(scale_mapping)] + duplicates = unique(filter(p -> count(==(p), process_names) > 1, process_names)) + isempty(duplicates) && continue + duplicate_names = join(string.(duplicates), ", ") + error( + "Scale `$scale` defines duplicate process(es): $duplicate_names. ", + "Keep only one model per process at a given scale (or use hard dependencies)." + ) + end +end + +function _process_name_for_mapping_check(model) + try + return process(model) + catch + return Symbol(nameof(typeof(model))) + end +end + +function _check_mapped_sources_exist!(mapping::Dict{String,Tuple}) + available_variables = _available_variables_by_scale(mapping) + scale_rates = _declared_model_rates_by_scale(mapping) + + for (target_scale, scale_mapping) in mapping + for item in scale_mapping + mapped_vars = _mapping_item_mapped_variables(item) + isempty(mapped_vars) && continue + + base_model = _mapping_item_model(item) + model_inputs = Set(keys(inputs_(base_model))) + model_outputs = Set(keys(outputs_(base_model))) + for mapped_var in mapped_vars + mapped_variable_name = first(mapped_var) isa PreviousTimeStep ? first(mapped_var).variable : first(mapped_var) + checks_source_value = (mapped_variable_name in model_inputs) && !(mapped_variable_name in model_outputs) + + for (source_scale_raw, source_variable) in _as_mapping_sources(last(mapped_var)) + source_scale = isempty(source_scale_raw) ? target_scale : source_scale_raw + + haskey(mapping, source_scale) || error( + "Scale `$target_scale` maps variable `$(first(mapped_var))` to missing scale `$source_scale`. ", + "Add `$source_scale` to ModelMapping, or fix the mapped scale name." + ) + + if checks_source_value && source_variable ∉ available_variables[source_scale] + error( + "Scale `$target_scale` maps variable `$(first(mapped_var))` to `$source_scale.$source_variable`, ", + "but `$source_variable` is not available at scale `$source_scale` (neither model output nor Status variable). ", + "Define a model output for `$source_variable`, initialize it in the source scale Status, or update the mapping." + ) + end + + if checks_source_value && !_rates_compatible(scale_rates[target_scale], scale_rates[source_scale]) + error( + "Scale `$target_scale` declares model rate $(scale_rates[target_scale]) but maps input `$(first(mapped_var))` ", + "from scale `$source_scale` with model rate $(scale_rates[source_scale]). ", + "Align model rates between scales or remove explicit `model_rate` declarations." + ) + end + end + end + end + end +end + +_mapping_item_mapped_variables(item::ModelSpec) = mapped_variables_(item) +_mapping_item_mapped_variables(item::MultiScaleModel) = mapped_variables_(item) +_mapping_item_mapped_variables(::Any) = Pair{Symbol,String}[] + +_mapping_item_model(item::ModelSpec) = model_(item) +_mapping_item_model(item::MultiScaleModel) = model_(item) + +_as_mapping_sources(source::Pair{<:AbstractString,Symbol}) = (String(first(source)) => last(source),) +_as_mapping_sources(source::AbstractVector{<:Pair{<:AbstractString,Symbol}}) = + Tuple(String(first(item)) => last(item) for item in source) + +function _available_variables_by_scale(mapping::Dict{String,Tuple}) + available = Dict{String,Set{Symbol}}() + for (scale, scale_mapping) in mapping + vars = Set{Symbol}() + for model in get_models(scale_mapping) + union!(vars, keys(outputs_(model))) + end + st = get_status(scale_mapping) + !isnothing(st) && union!(vars, keys(st)) + available[scale] = vars + end + return available +end + +function _declared_model_rates_by_scale(mapping::Dict{String,Tuple}) + rates = Dict{String,Any}() + for (scale, scale_mapping) in mapping + declared_rates = unique(filter(!isnothing, map(model_rate, get_models(scale_mapping)))) + if length(declared_rates) > 1 + error( + "Scale `$scale` declares incompatible model rates $(declared_rates). ", + "Use a single rate per scale, or leave `model_rate` undefined (`nothing`)." + ) + end + rates[scale] = isempty(declared_rates) ? nothing : only(declared_rates) + end + return rates +end + +_rates_compatible(rate1, rate2) = isnothing(rate1) || isnothing(rate2) || rate1 == rate2 diff --git a/src/mtg/mapping/model_generation_from_status_vectors.jl b/src/mtg/mapping/model_generation_from_status_vectors.jl index bd894673a..b85291cbe 100644 --- a/src/mtg/mapping/model_generation_from_status_vectors.jl +++ b/src/mtg/mapping/model_generation_from_status_vectors.jl @@ -52,7 +52,7 @@ function replace_mapping_status_vectors_with_generated_models(mapping_with_vecto (organ, check) = check_statuses_contain_no_remaining_vectors(mapping_with_vectors_in_status) if check @warn "No vectors, or types deriving from AbstractVector found in statuses, returning mapping as is." - return mapping_with_vectors_in_status + return mapping_with_vectors_in_status isa ModelMapping ? mapping_with_vectors_in_status : ModelMapping(mapping_with_vectors_in_status) end # we are now certain a model will be generated, and that the timestep models need to be inserted @@ -101,7 +101,7 @@ function replace_mapping_status_vectors_with_generated_models(mapping_with_vecto end end - return mapping + return ModelMapping(mapping) end # Note : eval works in global scope, and state synchronisation doesn't occur until one returns to top-level @@ -251,7 +251,11 @@ function modellist_to_mapping(modellist_original::ModelList, modellist_status; n # TODO sanity check end - return mtg, mapping, Dict(default_scale => all_vars) + return mtg, ModelMapping(mapping), Dict(default_scale => all_vars) +end + +function modellist_to_mapping(mapping::ModelMapping{SingleScale}, modellist_status; nsteps=nothing, outputs=nothing) + modellist_to_mapping(mapping.data, modellist_status; nsteps=nsteps, outputs=outputs) end function check_statuses_contain_no_remaining_vectors(mapping) diff --git a/src/mtg/mapping/reverse_mapping.jl b/src/mtg/mapping/reverse_mapping.jl index 059b42947..00faadd73 100644 --- a/src/mtg/mapping/reverse_mapping.jl +++ b/src/mtg/mapping/reverse_mapping.jl @@ -1,5 +1,6 @@ """ - reverse_mapping(mapping::Dict{String,Tuple{Any,Vararg{Any}}}; all=true) + reverse_mapping(mapping::ModelMapping; all=true) + reverse_mapping(mapping::AbstractDict{String,Tuple{Any,Vararg{Any}}}; all=true) reverse_mapping(mapped_vars::Dict{String,Dict{Symbol,Any}}) Get the reverse mapping of a dictionary of model mapping, *i.e.* the variables that are mapped to other scales, or in other words, @@ -8,7 +9,7 @@ This is used for *e.g.* knowing which scales are needed to add values to others. # Arguments -- `mapping::Dict{String,Any}`: A dictionary of model mapping. +- `mapping::ModelMapping` (or dictionary-like mapping): the model mapping. - `all::Bool`: Whether to get all the variables that are mapped to other scales, including the ones that are mapped as single values. # Returns @@ -30,7 +31,7 @@ julia> using PlantSimEngine.Examples; ``` ```jldoctest mylabel -julia> mapping = Dict( \ +julia> mapping = ModelMapping( \ "Plant" => \ MultiScaleModel( \ model=ToyCAllocationModel(), \ @@ -67,7 +68,7 @@ Dict{String, Dict{String, Dict{Symbol, Any}}} with 3 entries: "Leaf" => Dict("Plant"=>Dict(:carbon_allocation=>:carbon_allocation, :ca… ``` """ -function reverse_mapping(mapping::Dict{String,T}; all=true) where {T<:Any} +function reverse_mapping(mapping::AbstractDict{String,T}; all=true) where {T<:Any} # Method for the reverse mapping applied directly on the mapping (not used in the code base) mapped_vars = mapped_variables(mapping, first(hard_dependencies(mapping; verbose=false)), verbose=false) reverse_mapping(mapped_vars, all=all) @@ -101,4 +102,4 @@ function reverse_mapping(mapped_vars::Dict{String,Dict{Symbol,Any}}; all=true) end end filter!(x -> length(last(x)) > 0, reverse_multiscale_mapping) -end \ No newline at end of file +end diff --git a/src/mtg/save_results.jl b/src/mtg/save_results.jl index 51baffcbd..df585215e 100644 --- a/src/mtg/save_results.jl +++ b/src/mtg/save_results.jl @@ -34,7 +34,7 @@ julia> using PlantSimEngine.Examples; Define the models mapping: ```jldoctest mylabel -julia> mapping = Dict( \ +julia> mapping = ModelMapping( \ "Plant" => ( \ MultiScaleModel( \ model=ToyCAllocationModel(), \ diff --git a/src/processes/model_initialisation.jl b/src/processes/model_initialisation.jl index 95a8cc762..0bebd69d2 100755 --- a/src/processes/model_initialisation.jl +++ b/src/processes/model_initialisation.jl @@ -1,8 +1,9 @@ """ to_initialize(; verbose=true, vars...) - to_initialize(m::T) where T <: ModelList + to_initialize(m::T) where T <: ModelMapping to_initialize(m::DependencyGraph) - to_initialize(mapping::Dict{String,T}, graph=nothing) + to_initialize(mapping::ModelMapping, graph=nothing) + to_initialize(mapping::AbstractDict{String,T}, graph=nothing) Return the variables that must be initialized providing a set of models and processes. The function takes into account model coupling and only returns the variables that are needed @@ -12,9 +13,9 @@ considering that some variables that are outputs of some models are used as inpu - `verbose`: if `true`, print information messages. - `vars...`: the models and processes to consider. -- `m::T`: a [`ModelList`](@ref). +- `m::T`: a [`ModelMapping`](@ref). - `m::DependencyGraph`: a [`DependencyGraph`](@ref). -- `mapping::Dict{String,T}`: a mapping that associates models to organs. +- `mapping::ModelMapping` (or dictionary-like mapping): associates models to organs/scales. - `graph`: a graph representing a plant or a scene, *e.g.* a multiscale tree graph. The graph is used to check if variables that are not initialized can be found in the graph nodes attributes. @@ -29,10 +30,10 @@ using PlantSimEngine.Examples to_initialize(process1=Process1Model(1.0), process2=Process2Model()) # Or using a component directly: -models = ModelList(process1=Process1Model(1.0), process2=Process2Model()) +models = ModelMapping(process1=Process1Model(1.0), process2=Process2Model()) to_initialize(models) -m = ModelList( +m = ModelMapping( ( process1=Process1Model(1.0), process2=Process2Model() @@ -51,13 +52,13 @@ using PlantSimEngine # Load the dummy models given as example in the package: using PlantSimEngine.Examples -mapping = Dict( - "Leaf" => ModelList( +mapping = ModelMapping( + "Leaf" => ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model() ), - "Internode" => ModelList( + "Internode" => ModelMapping( process1=Process1Model(1.0), ) ) @@ -108,12 +109,12 @@ end #Return the variables that must be initialized providing a set of models and processes. The #function just returns the inputs and outputs of each model, with their default values. #To take into account model coupling, use the function at an upper-level instead, *i.e.* -# `to_initialize(m::ModelList)` or `to_initialize(m::DependencyGraph)`. +# `to_initialize(m::ModelMapping)` or `to_initialize(m::DependencyGraph)`. function to_initialize(m::AbstractDependencyNode) return (inputs=inputs_(m.value), outputs=outputs_(m.value)) end -function to_initialize(m::T) where {T<:Dict{String,ModelList}} +function to_initialize(m::T) where {T<:Dict{String,ModelMapping}} toinit = Dict{String,NamedTuple}() for (key, value) in m # key = "Leaf"; value = m[key] @@ -139,7 +140,7 @@ function to_initialize(; verbose=true, vars...) end # For the list of mapping given to an MTG: -function to_initialize(mapping::Dict{String,T}, graph=nothing) where {T} +function to_initialize(mapping::AbstractDict{String,T}, graph=nothing) where {T} # Get the variables in the MTG: if isnothing(graph) vars_in_mtg = Symbol[] @@ -163,8 +164,8 @@ function to_initialize(mapping::Dict{String,T}, graph=nothing) where {T} end """ - init_status!(object::Dict{String,ModelList};vars...) - init_status!(component::ModelList;vars...) + init_status!(object::Dict{String,ModelMapping};vars...) + init_status!(component::ModelMapping;vars...) Initialise model variables for components with user input. @@ -177,12 +178,12 @@ using PlantSimEngine using PlantSimEngine.Examples models = Dict( - "Leaf" => ModelList( + "Leaf" => ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model() ), - "InterNode" => ModelList( + "InterNode" => ModelMapping( process1=Process1Model(1.0), ) ) @@ -191,7 +192,7 @@ init_status!(models, var1=1.0 , var2=2.0) status(models["Leaf"]) ``` """ -function init_status!(object::Dict{String,ModelList}; vars...) +function init_status!(object::Dict{String,ModelMapping}; vars...) new_vals = (; vars...) for (component_name, component) in object @@ -205,7 +206,7 @@ function init_status!(object::Dict{String,ModelList}; vars...) end end -function init_status!(component::T; vars...) where {T<:ModelList} +function init_status!(component::T; vars...) where {T<:ModelMapping} new_vals = (; vars...) for j in keys(new_vals) if !in(j, keys(component.status)) @@ -269,8 +270,8 @@ function init_variables(models::T; verbose::Bool=true) where {T<:NamedTuple} end """ - is_initialized(m::T) where T <: ModelList - is_initialized(m::T, models...) where T <: ModelList + is_initialized(m::T) where T <: ModelMapping + is_initialized(m::T, models...) where T <: ModelMapping Check if the variables that must be initialized are, and return `true` if so, and `false` and an information message if not. @@ -290,7 +291,7 @@ using PlantSimEngine # Load the dummy models given as example in the package: using PlantSimEngine.Examples -models = ModelList( +models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model() @@ -365,7 +366,7 @@ using PlantSimEngine # Load the dummy models given as example in the package: using PlantSimEngine.Examples -models = ModelList( +models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model() diff --git a/src/processes/models_inputs_outputs.jl b/src/processes/models_inputs_outputs.jl index d62ed46b4..f8b6d5ac9 100644 --- a/src/processes/models_inputs_outputs.jl +++ b/src/processes/models_inputs_outputs.jl @@ -119,11 +119,12 @@ Defaults to `nothing` (runtime falls back to `PlantMeteo.RollingWindow()` behavi meteo_window(spec::ModelSpec) = spec.meteo_window """ - inputs(mapping::Dict{String,T}) + inputs(mapping::ModelMapping) + inputs(mapping::AbstractDict{String,T}) Get the inputs of the models in a mapping, for each process and organ type. """ -function inputs(mapping::Dict{String,T}) where {T} +function inputs(mapping::AbstractDict{String,T}) where {T} vars = Dict{String,NamedTuple}() for organ in keys(mapping) mods = pairs(parse_models(get_models(mapping[organ]))) @@ -164,11 +165,12 @@ function outputs(v::T, vars...) where {T<:AbstractModel} end """ - outputs(mapping::Dict{String,T}) + outputs(mapping::ModelMapping) + outputs(mapping::AbstractDict{String,T}) Get the outputs of the models in a mapping, for each process and organ type. """ -function outputs(mapping::Dict{String,T}) where {T} +function outputs(mapping::AbstractDict{String,T}) where {T} vars = Dict{String,NamedTuple}() for organ in keys(mapping) mods = pairs(parse_models(get_models(mapping[organ]))) @@ -259,12 +261,13 @@ function variables(pkg::Module) end """ - variables(mapping::Dict{String,T}) + variables(mapping::ModelMapping) + variables(mapping::AbstractDict{String,T}) Get the variables (inputs and outputs) of the models in a mapping, for each process and organ type. """ -function variables(mapping::Dict{String,T}) where {T} +function variables(mapping::AbstractDict{String,T}) where {T} vars = Dict{String,NamedTuple}() for organ in keys(mapping) mods = pairs(parse_models(get_models(mapping[organ]))) diff --git a/src/processes/process_generation.jl b/src/processes/process_generation.jl index f1db9e5be..9bd1e90e8 100644 --- a/src/processes/process_generation.jl +++ b/src/processes/process_generation.jl @@ -143,10 +143,10 @@ macro process(f, args...) end ``` - Note that {#8abeff}run!(){/#8abeff} takes six arguments: the model type (used for dispatch), the ModelList, the status, the meteorology, + Note that {#8abeff}run!(){/#8abeff} takes six arguments: the model type (used for dispatch), the ModelMapping, the status, the meteorology, the constants and any extra values. - Then we can use variables from the status as inputs or outputs, model parameters from the ModelList (indexing by process, here + Then we can use variables from the status as inputs or outputs, model parameters from the ModelMapping (indexing by process, here using "$(process_name)" as the process name), and meteorology variables. Note that our example model has an hard-dependency on another process called `other_process_name` that is called using the {#8abeff}run!(){/#8abeff} function with @@ -169,10 +169,10 @@ macro process(f, args...) !!! tip "Variables and parameters usage" Note that {#8abeff}run!(){/#8abeff} takes six arguments: the model type (used - for dispatch), the ModelList, the status, the meteorology, the constants and + for dispatch), the ModelMapping, the status, the meteorology, the constants and any extra values. Then we can use variables from the status as inputs or outputs, model parameters - from the ModelList (indexing by process, here using "$(process_name)" as the + from the ModelMapping (indexing by process, here using "$(process_name)" as the process name), and meteorology variables. """ ) diff --git a/src/run.jl b/src/run.jl index d19414689..374a20e68 100644 --- a/src/run.jl +++ b/src/run.jl @@ -9,7 +9,7 @@ If several time-steps are given, the models are run sequentially for each time-s # Arguments -- `object`: a [`ModelList`](@ref), an array or dict of `ModelList`, or a plant graph (MTG). +- `object`: a [`ModelMapping`](@ref) for single-scale runs, or a plant graph (MTG) for multiscale runs. - `meteo`: a [`PlantMeteo.TimeStepTable`](https://palmstudio.github.io/PlantMeteo.jl/stable/API/#PlantMeteo.TimeStepTable) of [`PlantMeteo.Atmosphere`](https://palmstudio.github.io/PlantMeteo.jl/stable/API/#PlantMeteo.Atmosphere) or a single `PlantMeteo.Atmosphere`. - `constants`: a [`PlantMeteo.Constants`](https://palmstudio.github.io/PlantMeteo.jl/stable/API/#PlantMeteo.Constants) object, or a `NamedTuple` of constant keys and values. @@ -17,7 +17,7 @@ If several time-steps are given, the models are run sequentially for each time-s - `check`: if `true`, check the validity of the model list before running the simulation (takes a little bit of time), and return more information while running. - `executor`: the [`Floops`](https://juliafolds.github.io/FLoops.jl/stable/) executor used to run the simulation either in sequential (`executor=SequentialEx()`), in a multi-threaded way (`executor=ThreadedEx()`, the default), or in a distributed way (`executor=DistributedEx()`). -- `mapping`: a mapping between the MTG and the model list. +- `mapping`: a [`ModelMapping`](@ref) between MTG scales and models. - `nsteps`: the number of time-steps to run, only needed if no meteo is given (else it is infered from it). - `outputs`: the outputs to get in dynamic for each node type of the MTG. - `multirate`: experimental feature flag enabling temporal stream-based input resolution for multiscale simulations. @@ -66,10 +66,10 @@ Load the dummy models given as example in the `Examples` sub-module: julia> using PlantSimEngine.Examples; ``` -Create a model list: +Create a model mapping: ```jldoctest run -julia> models = ModelList(Process1Model(1.0), Process2Model(), Process3Model(), status = (var1=1.0, var2=2.0)); +julia> mapping = ModelMapping(Process1Model(1.0), Process2Model(), Process3Model(); status = (var1=1.0, var2=2.0)); ``` Create meteo data: @@ -81,7 +81,7 @@ julia> meteo = Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65, Ri_PAR_f=300.0); Run the simulation: ```jldoctest run -julia> outputs_sim = run!(models, meteo); +julia> outputs_sim = run!(mapping, meteo); ``` Get the results: @@ -117,6 +117,77 @@ function adjust_weather_timesteps_to_given_length(desired_length, meteo) end +function _all_modellists_collection(object) + if isa(object, AbstractArray) + return all(x -> x isa ModelList || x isa ModelMapping{SingleScale}, object) + elseif isa(object, AbstractDict) + return all(x -> x isa ModelList || x isa ModelMapping{SingleScale}, values(object)) + end + return false +end + +function _error_if_multirate_singlescale(multirate) + multirate || return nothing + error( + "`multirate=true` is only supported for MTG-based multiscale runs. ", + "For one scale, build a one-scale MTG and call `run!(mtg, mapping, ...; multirate=true)`." + ) +end + +_single_scale_runtime_object(object) = object +_single_scale_runtime_object(mapping::ModelMapping) = _modellist_from_model_mapping(mapping) + +function _modellist_from_model_mapping(mapping::ModelMapping{SingleScale}) + mapping.data +end + +function _modellist_from_model_mapping(::ModelMapping{MultiScale}) + error("This `ModelMapping` is a multiscale mapping. ", "Use `run!(mtg, mapping, ...)` for multiscale mappings.") +end + +_modellist_from_model_mapping(mapping::ModelList) = mapping + +function run!( + mapping::M, + meteo=nothing, + constants=PlantMeteo.Constants(), + extra=nothing; + tracked_outputs=nothing, + check=true, + executor=ThreadedEx(), + multirate=false, + return_requested_outputs=false, + requested_outputs_sink=DataFrames.DataFrame +) where {M<:Union{ModelMapping{SingleScale},ModelList}} + _error_if_multirate_singlescale(multirate) + model_list = _modellist_from_model_mapping(mapping) + _run_modellist_singleton( + model_list, + meteo, + constants, + extra; + tracked_outputs=tracked_outputs, + check=check, + executor=executor, + return_requested_outputs=return_requested_outputs + ) +end + +function run!( + ::ModelMapping{MultiScale}, + meteo=nothing, + constants=PlantMeteo.Constants(), + extra=nothing; + tracked_outputs=nothing, + check=true, + executor=ThreadedEx(), + multirate=false, + return_requested_outputs=false, + requested_outputs_sink=DataFrames.DataFrame +) + error("This `ModelMapping` is a multiscale mapping. ", "Use `run!(mtg, mapping, ...)` for multiscale mappings.") +end + # User entry point, which uses traits to dispatch to the correct method. # The traits are defined in table_traits.jl # and define either TableAlike, TreeAlike or SingletonAlike objects. @@ -165,6 +236,13 @@ function run!( return_requested_outputs=false, requested_outputs_sink=DataFrames.DataFrame ) where {T<:Union{AbstractArray,AbstractDict},A} + _error_if_multirate_singlescale(multirate) + if _all_modellists_collection(object) + Base.depwarn( + "`run!` with a collection of `ModelList` is deprecated. Use a collection of `ModelMapping` objects instead.", + :run! + ) + end tracked_outputs isa OutputRequest && error("`OutputRequest` is only supported for MTG multi-rate simulations.") tracked_outputs isa AbstractVector{<:OutputRequest} && error("`OutputRequest` is only supported for MTG multi-rate simulations.") @@ -206,6 +284,61 @@ function run!( return_requested_outputs=false, requested_outputs_sink=DataFrames.DataFrame ) where {T<:ModelList} + _error_if_multirate_singlescale(multirate) + Base.depwarn( + "`run!(::ModelList, ...)` is deprecated. Use `run!(ModelMapping(...), ...)` instead.", + :run! + ) + _run_modellist_singleton( + object, + meteo, + constants, + extra; + tracked_outputs=tracked_outputs, + check=check, + executor=executor, + return_requested_outputs=return_requested_outputs + ) +end + +function run!( + ::SingletonAlike, + object::T, + meteo=nothing, + constants=PlantMeteo.Constants(), + extra=nothing; + tracked_outputs=nothing, + check=true, + executor=ThreadedEx(), + multirate=false, + return_requested_outputs=false, + requested_outputs_sink=DataFrames.DataFrame +) where {T<:ModelMapping{SingleScale}} + _error_if_multirate_singlescale(multirate) + model_list = _modellist_from_model_mapping(object) + + _run_modellist_singleton( + model_list, + meteo, + constants, + extra; + tracked_outputs=tracked_outputs, + check=check, + executor=executor, + return_requested_outputs=return_requested_outputs + ) +end + +function _run_modellist_singleton( + object::ModelList, + meteo=nothing, + constants=PlantMeteo.Constants(), + extra=nothing; + tracked_outputs=nothing, + check=true, + executor=ThreadedEx(), + return_requested_outputs=false +) tracked_outputs isa OutputRequest && error("`OutputRequest` is only supported for MTG multi-rate simulations.") tracked_outputs isa AbstractVector{<:OutputRequest} && error("`OutputRequest` is only supported for MTG multi-rate simulations.") @@ -304,12 +437,19 @@ function run!( return_requested_outputs=false, requested_outputs_sink=DataFrames.DataFrame ) where {T<:Union{AbstractArray,AbstractDict}} + _error_if_multirate_singlescale(multirate) + if _all_modellists_collection(object) + Base.depwarn( + "`run!` with a collection of `ModelList` is deprecated. Use a collection of `ModelMapping` objects instead.", + :run! + ) + end tracked_outputs isa OutputRequest && error("`OutputRequest` is only supported for MTG multi-rate simulations.") tracked_outputs isa AbstractVector{<:OutputRequest} && error("`OutputRequest` is only supported for MTG multi-rate simulations.") return_requested_outputs && error("`return_requested_outputs=true` is only supported for MTG multi-rate simulations.") - - dep_graphs = [dep(obj) for obj in collect(values(object))] + runtime_objects = [_single_scale_runtime_object(obj) for obj in collect(values(object))] + dep_graphs = [dep(obj) for obj in runtime_objects] #obj_parallelizable = all([object_parallelizable(graph) for graph in dep_graphs]) # Check if the simulation can be parallelized over objects: @@ -320,7 +460,7 @@ function run!( end # Each object: - for (i, obj) in enumerate(collect(values(object))) + for (i, obj) in enumerate(runtime_objects) if check # Check if the meteo data and the status have the same length (or length 1) @@ -403,7 +543,7 @@ end function run!( object::MultiScaleTreeGraph.Node, - mapping::Dict{String,T} where {T}, + mapping::ModelMapping, meteo=nothing, constants=PlantMeteo.Constants(), extra=nothing; @@ -420,6 +560,8 @@ function run!( # Keep TimeStepTable intact in MTG multi-rate runs so model-clock meteo # sampling/aggregation can use PlantMeteo sampler APIs. meteo + elseif DataFormat(meteo) == TableAlike() + nsteps == 1 ? Tables.rows(meteo)[1] : meteo else adjust_weather_timesteps_to_given_length(nsteps, meteo) end @@ -450,6 +592,40 @@ function run!( return outputs(sim) end +function run!( + object::MultiScaleTreeGraph.Node, + mapping::AbstractDict{String,T} where {T}, + meteo=nothing, + constants=PlantMeteo.Constants(), + extra=nothing; + nsteps=nothing, + tracked_outputs=nothing, + check=true, + executor=ThreadedEx(), + multirate=false, + return_requested_outputs=false, + requested_outputs_sink=DataFrames.DataFrame +) + Base.depwarn( + "`run!(mtg, mapping::AbstractDict, ...)` is deprecated. Use `run!(mtg, ModelMapping(mapping), ...)` or construct `ModelMapping(...)` directly.", + :run! + ) + run!( + object, + ModelMapping(mapping), + meteo, + constants, + extra; + nsteps=nsteps, + tracked_outputs=tracked_outputs, + check=check, + executor=executor, + multirate=multirate, + return_requested_outputs=return_requested_outputs, + requested_outputs_sink=requested_outputs_sink + ) +end + function run!( ::TreeAlike, object::GraphSimulation, diff --git a/src/traits/table_traits.jl b/src/traits/table_traits.jl index 2e6373273..480e31ce0 100644 --- a/src/traits/table_traits.jl +++ b/src/traits/table_traits.jl @@ -57,6 +57,7 @@ DataFormat(::Type{<:Dict}) = TableAlike() DataFormat(::Type{<:NamedTuple}) = SingletonAlike() DataFormat(::Type{<:Status}) = SingletonAlike() DataFormat(::Type{<:ModelList{Mo,S} where {Mo,S}}) = SingletonAlike() +DataFormat(::Type{<:ModelMapping{SingleScale}}) = SingletonAlike() DataFormat(::Type{<:GraphSimulation}) = TreeAlike() DataFormat(::Type{<:PlantMeteo.AbstractAtmosphere}) = SingletonAlike() @@ -64,4 +65,4 @@ DataFormat(::Type{<:PlantMeteo.TimeStepRow}) = SingletonAlike() DataFormat(::Type{<:Nothing}) = SingletonAlike() # For meteo == Nothing DataFormat(T::Type{<:Any}) = error("Unknown data format: $T.\nPlease define a `DataFormat` method, e.g.: DataFormat(::Type{$T}) method.") DataFormat(x::T) where {T} = DataFormat(T) -DataFormat(::Type{<:DataFrames.DataFrameRow}) = SingletonAlike() \ No newline at end of file +DataFormat(::Type{<:DataFrames.DataFrameRow}) = SingletonAlike() diff --git a/test/helper-functions.jl b/test/helper-functions.jl index 7f6aafa62..540329e7b 100644 --- a/test/helper-functions.jl +++ b/test/helper-functions.jl @@ -1,28 +1,3 @@ -# Simple helper functions that can be used in various tests here and there - -function compare_outputs_modellist_mapping(filtered_outputs, graphsim) - outputs_df_dict = convert_outputs(graphsim.outputs, DataFrame) - @assert length(outputs_df_dict) == 1 - - outputs_df = last(first(outputs_df_dict)) - outputs_df_outputs_only = select(outputs_df, Not([:timestep#=, :organ=#, :node])) - models_df = DataFrame(filtered_outputs) - - models_df_sorted = models_df[:, sortperm(names(models_df))] - outputs_df_outputs_only_sorted = outputs_df_outputs_only[:, sortperm(names(outputs_df_outputs_only))] - return outputs_df_outputs_only_sorted == models_df_sorted -end - -# doesn't check for mtg equality -function compare_outputs_graphsim_old(graphsim, graphsim2) - outputs_df = convert_outputs(graphsim.outputs, DataFrame) - outputs_df_sorted = outputs_df[:, sortperm(names(outputs_df))] - - outputs2_df = convert_outputs(graphsim2.outputs, DataFrame) - outputs2_df_sorted = outputs2_df[:, sortperm(names(outputs2_df))] - return outputs_df_sorted == outputs2_df_sorted -end - # doesn't check for mtg equality function compare_outputs_graphsim(graphsim, graphsim2) outputs_df_dict = convert_outputs(graphsim.outputs, DataFrame) @@ -40,23 +15,42 @@ function compare_outputs_graphsim(graphsim, graphsim2) return false end end - + return true end function compare_outputs_modellists(filtered_outputs_1, filtered_outputs_2) - models_df_1 = DataFrame(filtered_outputs_1) + models_df_1 = DataFrame(filtered_outputs_1) models_df_sorted_1 = models_df_1[:, sortperm(names(models_df_1))] - models_df_2 = DataFrame(filtered_outputs_2) + models_df_2 = DataFrame(filtered_outputs_2) models_df_sorted_2 = models_df_2[:, sortperm(names(models_df_2))] return models_df_sorted_2 == models_df_sorted_1 end +function compare_outputs_modellist_mapping(filtered_outputs_modellist, graphsim) + modellist_df = DataFrame(filtered_outputs_modellist) + modellist_sorted = modellist_df[:, sortperm(names(modellist_df))] + + outputs_df = convert_outputs(graphsim.outputs, DataFrame) + @assert haskey(outputs_df, "Default") + common_cols = filter(c -> c in names(outputs_df["Default"]), names(modellist_sorted)) + mapping_sorted = outputs_df["Default"][:, common_cols] + modellist_sorted = modellist_sorted[:, common_cols] + + # Keep deterministic order in case columns are provided in different orders. + mapping_sorted = mapping_sorted[:, sortperm(names(mapping_sorted))] + modellist_sorted = modellist_sorted[:, sortperm(names(modellist_sorted))] + + return modellist_sorted == mapping_sorted +end + # Breaking this function into two to ensure eval() state synchronisation happens (see comments around the modellist_to_mapping definition) # Naming could be better -function check_multiscale_simulation_is_equivalent_begin(models::ModelList, status, meteo) - - mtg, mapping, out = PlantSimEngine.modellist_to_mapping(models, status; nsteps=length(meteo), outputs=nothing) +function check_multiscale_simulation_is_equivalent_begin(mapping::ModelMapping, meteo) + _, models_at_scale = only(pairs(mapping)) + status_nt = NamedTuple(something(PlantSimEngine.get_status(models_at_scale), Status())) + models = ModelMapping(PlantSimEngine.get_models(models_at_scale)...; status=status_nt) + mtg, mapping, out = PlantSimEngine.modellist_to_mapping(models, status_nt; nsteps=length(meteo), outputs=nothing) return mtg, mapping, out end @@ -69,16 +63,16 @@ function check_multiscale_simulation_is_equivalent_end(modellist_outputs, mtg, m nothing; check=true, executor=SequentialEx() - ); + ) return compare_outputs_modellist_mapping(modellist_outputs, graph_sim) end # Quick and naive first version. Doesn't check if everything is timestep parallelizable, doesn't check for nthreads etc. -function run_single_and_multi_thread_modellist(modellist, tracked_outputs, meteo) - out_seq = run!(modellist, meteo; tracked_outputs = tracked_outputs, executor = SequentialEx()) - modellist_mt = copy(modellist) - out_mt = run!(modellist_mt, meteo; tracked_outputs = tracked_outputs, executor = ThreadedEx()) +function run_single_and_multi_thread_modellist(mapping::ModelMapping, tracked_outputs, meteo) + out_seq = run!(mapping, meteo; tracked_outputs=tracked_outputs, executor=SequentialEx()) + mapping_mt = copy(mapping) + out_mt = run!(mapping_mt, meteo; tracked_outputs=tracked_outputs, executor=ThreadedEx()) return out_seq, out_mt end @@ -96,226 +90,276 @@ function get_simple_meteo_bank() df = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) - meteos= - [#=nothing,=# Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65, Ri_PAR_f=300.0), - Weather([Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65)]), - Weather( - [ - Atmosphere(T=20.0, Wind=1.0, Rh=0.65, Ri_PAR_f=300.0), - Atmosphere(T=25.0, Wind=0.5, Rh=0.8, Ri_PAR_f=500.0) - ]), - - Weather([Atmosphere(T=20.0, Wind=1.0, Rh=0.65, Ri_PAR_f=200.0), - Atmosphere(T=18.0, Wind=1.0, Rh=0.65, Ri_PAR_f=100.0), - Atmosphere(T=19.0, Wind=1.0, Rh=0.65, Ri_PAR_f=200.0), - Atmosphere(T=30.0, Wind=0.5, Rh=0.6, Ri_PAR_f=100.0), - Atmosphere(T=20.0, Wind=1.0, Rh=0.6, Ri_PAR_f=200.0), - Atmosphere(T=25.0, Wind=1.0, Rh=0.6, Ri_PAR_f=200.0), - Atmosphere(T=10.0, Wind=0.5, Rh=0.6, Ri_PAR_f=200.0)]), - df, - df[1,:], - DataFrame(df[1,:]), - ] + meteos = + [Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65, Ri_PAR_f=300.0), #=nothing,=# + Weather([Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65)]), + Weather( + [ + Atmosphere(T=20.0, Wind=1.0, Rh=0.65, Ri_PAR_f=300.0), + Atmosphere(T=25.0, Wind=0.5, Rh=0.8, Ri_PAR_f=500.0) + ]), Weather([Atmosphere(T=20.0, Wind=1.0, Rh=0.65, Ri_PAR_f=200.0), + Atmosphere(T=18.0, Wind=1.0, Rh=0.65, Ri_PAR_f=100.0), + Atmosphere(T=19.0, Wind=1.0, Rh=0.65, Ri_PAR_f=200.0), + Atmosphere(T=30.0, Wind=0.5, Rh=0.6, Ri_PAR_f=100.0), + Atmosphere(T=20.0, Wind=1.0, Rh=0.6, Ri_PAR_f=200.0), + Atmosphere(T=25.0, Wind=1.0, Rh=0.6, Ri_PAR_f=200.0), + Atmosphere(T=10.0, Wind=0.5, Rh=0.6, Ri_PAR_f=200.0)]), + df, + df[1, :], + DataFrame(df[1, :]), + ] return meteos end function get_modellist_bank() meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) - + rue = 0.3 vals = (var1=15.0, var2=0.3)#, TT_cu=cumsum(meteo_day.TT)) vals2 = (TT_cu=cumsum(meteo_day.TT),) vals3 = (var1=15.0, var2=0.3) - vals4 = (var9 =1.0,var0=1.0) + vals4 = (var9=1.0, var0=1.0) vals5 = (var0=1.0,) vals6 = (var0=1.0,) - - status_tuples = [vals, vals2, vals3, vals4, vals5, vals6] - models = [ModelList( - process1=Process1Model(1.0), - process2=Process2Model(), - status=vals - ), - ModelList( - ToyLAIModel(), - Beer(0.5), - ToyRUEGrowthModel(rue), - status=vals2, - ), - - ModelList( - process1=Process1Model(1.0), - process2=Process2Model(), - process3=Process3Model(), - status=vals3 - ), - - ModelList( - process1=Process1Model(1.0), - process2=Process2Model(), - process3=Process3Model(), - process4=Process4Model(), - process5=Process5Model(), - process6=Process6Model(), - # process7=Process7Model(), - status=vals4 - ), - - ModelList( - process1=Process1Model(1.0), - process2=Process2Model(), - process3=Process3Model(), - process4=Process4Model(), - process5=Process5Model(), - process6=Process6Model(), - process7=Process7Model(), - status=vals5 - ), - - ModelList( - process1=Process1Model(1.0), - process2=Process2Model(), - process3=Process3Model(), - process4=Process4Model(), - process5=Process5Model(), - status=vals6 - ), + status_tuples = [vals, vals2, vals3, vals4, vals5, vals6] - ] + models = [ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + status=vals + ), + ModelMapping( + ToyLAIModel(), + Beer(0.5), + ToyRUEGrowthModel(rue), + status=vals2, + ), ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + status=vals3 + ), ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + process4=Process4Model(), + process5=Process5Model(), + process6=Process6Model(), + # process7=Process7Model(), + status=vals4 + ), ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + process4=Process4Model(), + process5=Process5Model(), + process6=Process6Model(), + process7=Process7Model(), + status=vals5 + ), ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + process4=Process4Model(), + process5=Process5Model(), + status=vals6 + ),] + + outputs_tuples_vectors = + [ + # this one has one tuple with a duplicate, and one with a nonexistent variable + [NamedTuple(), (:var1,), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var5), #=(:var1, :var1),=# + (:var1, :var2, :var3, :var4, :var5)], #=(:var2, :var7, :var3, :var1),=# + [NamedTuple(), (:TT_cu,), (:TT_cu, :LAI), (:biomass, :LAI), (:TT_cu, :LAI, :aPPFD, :biomass, :biomass_increment),], [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), + (:var1, :var2, :var3, :var4, :var5, :var6)], #=(:var2, :var7, :var3, :var1),=# + [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), + (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6)], [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), + (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6), (:var1, :var2, :var3, :var4, :var5, :var6, :var7, :var8, :var9)], [NamedTuple(), (:var1,), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), #=(:var1, :var1),=# + (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6),], #=(:var1, :var2, :var3, :var4, :var5, :var6, :var7, :var8, :var9, :var0)=# + ] - outputs_tuples_vectors = - [ - # this one has one tuple with a duplicate, and one with a nonexistent variable - [NamedTuple(), (:var1,), #=(:var1, :var1),=# (:var1, :var2), (:var1, :var3), (:var1, :var4, :var5), - #=(:var2, :var7, :var3, :var1),=# (:var1, :var2, :var3, :var4, :var5)], + return models, status_tuples, outputs_tuples_vectors +end - [NamedTuple(), (:TT_cu,), (:TT_cu,:LAI) , (:biomass,:LAI), (:TT_cu, :LAI, :aPPFD, :biomass, :biomass_increment),], +function get_modelmapping_bank() + meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) - [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), - #=(:var2, :var7, :var3, :var1),=# (:var1, :var2, :var3, :var4, :var5, :var6)], + rue = 0.3 - [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), - (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6)], - - [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), - (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6) - , (:var1, :var2, :var3, :var4, :var5, :var6, :var7, :var8, :var9)], + vals = (var1=15.0, var2=0.3) + vals2 = (TT_cu=cumsum(meteo_day.TT),) + vals3 = (var1=15.0, var2=0.3) + vals4 = (var9=1.0, var0=1.0) + vals5 = (var0=1.0,) + vals6 = (var0=1.0,) - [NamedTuple(), (:var1,), #=(:var1, :var1),=# (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), - (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6) - , #=(:var1, :var2, :var3, :var4, :var5, :var6, :var7, :var8, :var9, :var0)=#], + status_tuples = [vals, vals2, vals3, vals4, vals5, vals6] + mappings = [ + ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + status=vals + ), + ModelMapping( + ToyLAIModel(), + Beer(0.5), + ToyRUEGrowthModel(rue); + status=vals2 + ), + ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + status=vals3 + ), + ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + process4=Process4Model(), + process5=Process5Model(), + process6=Process6Model(), + status=vals4 + ), + ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + process4=Process4Model(), + process5=Process5Model(), + process6=Process6Model(), + process7=Process7Model(), + status=vals5 + ), + ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + process4=Process4Model(), + process5=Process5Model(), + status=vals6 + ), ] - return models, status_tuples, outputs_tuples_vectors + outputs_tuples_vectors = + [ + [NamedTuple(), (:var1,), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var5), + (:var1, :var2, :var3, :var4, :var5)], [NamedTuple(), (:TT_cu,), (:TT_cu, :LAI), (:biomass, :LAI), (:TT_cu, :LAI, :aPPFD, :biomass, :biomass_increment),], [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), + (:var1, :var2, :var3, :var4, :var5, :var6)], [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), + (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6)], [NamedTuple(), (:var1,), (:var1, :var4), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), + (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6), (:var1, :var2, :var3, :var4, :var5, :var6, :var7, :var8, :var9)], [NamedTuple(), (:var1,), (:var1, :var2), (:var1, :var3), (:var1, :var4, :var6, :var5), + (:var2, :var7, :var3, :var1), (:var1, :var2, :var3, :var4, :var5, :var6)],] + + return mappings, status_tuples, outputs_tuples_vectors end # Could add some mtg variation too function get_simple_mapping_bank() mappings = [ - Dict( - "Scene" => ToyDegreeDaysCumulModel(), - "Plant" => ( - MultiScaleModel( - model=ToyLAIModel(), - mapped_variables=[:TT_cu => "Scene",],), - Beer(0.6), - MultiScaleModel( - model=ToyCAllocationModel(), - mapped_variables=[ - :carbon_assimilation => ["Leaf"], - :carbon_demand => ["Leaf", "Internode"], - :carbon_allocation => ["Leaf", "Internode"]],), - MultiScaleModel( - model=ToyPlantRmModel(), - mapped_variables=[:Rm_organs => ["Leaf" => :Rm, "Internode" => :Rm],],),), - "Internode" => ( - MultiScaleModel( - model=ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), - mapped_variables=[:TT => "Scene",],), - MultiScaleModel( - model=ToyInternodeEmergence(TT_emergence=20.0), - mapped_variables=[:TT_cu => "Scene"],), - ToyMaintenanceRespirationModel(1.5, 0.06, 25.0, 0.6, 0.004), - Status(carbon_biomass=1.0)), - "Leaf" => ( - MultiScaleModel( - model=ToyAssimModel(), - mapped_variables=[:soil_water_content => "Soil", :aPPFD => "Plant"],), - MultiScaleModel( - model=ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), - mapped_variables=[:TT => "Scene",],), - ToyMaintenanceRespirationModel(2.1, 0.06, 25.0, 1.0, 0.025), - Status(carbon_biomass=1.0)), - "Soil" => (ToySoilWaterModel(),),), -########## - Dict( - "Default" => ( - Process1Model(1.0), - Status(var1=15.0, var2=0.3,),),), -########## - Dict( - "Plant" => ( - MultiScaleModel( - model=ToyCAllocationModel(), - mapped_variables=[ - # inputs - :carbon_assimilation => ["Leaf"], - :carbon_demand => ["Leaf", "Internode"], - # outputs - :carbon_allocation => ["Leaf", "Internode"]],), - MultiScaleModel( - model=ToyPlantRmModel(), - mapped_variables=[:Rm_organs => ["Leaf" => :Rm, "Internode" => :Rm],],),), - "Internode" => ( - ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), - ToyMaintenanceRespirationModel(1.5, 0.06, 25.0, 0.6, 0.004), - Status(TT=10.0, carbon_biomass=1.0)), - "Leaf" => ( - MultiScaleModel( - model=ToyAssimModel(), - mapped_variables=[:soil_water_content => "Soil",], - # Notice we provide "Soil", not ["Soil"], so a single value is expected here - ), - ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), - Status(aPPFD=1300.0, TT=10.0, carbon_biomass=1.0), - ToyMaintenanceRespirationModel(2.1, 0.06, 25.0, 1.0, 0.025),), - "Soil" => (ToySoilWaterModel(),),), -################## + ModelMapping( + "Scene" => ToyDegreeDaysCumulModel(), + "Plant" => ( + MultiScaleModel( + model=ToyLAIModel(), + mapped_variables=[:TT_cu => "Scene",],), + Beer(0.6), + MultiScaleModel( + model=ToyCAllocationModel(), + mapped_variables=[ + :carbon_assimilation => ["Leaf"], + :carbon_demand => ["Leaf", "Internode"], + :carbon_allocation => ["Leaf", "Internode"]],), + MultiScaleModel( + model=ToyPlantRmModel(), + mapped_variables=[:Rm_organs => ["Leaf" => :Rm, "Internode" => :Rm],],),), + "Internode" => ( + MultiScaleModel( + model=ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), + mapped_variables=[:TT => "Scene",],), + MultiScaleModel( + model=ToyInternodeEmergence(TT_emergence=20.0), + mapped_variables=[:TT_cu => "Scene"],), + ToyMaintenanceRespirationModel(1.5, 0.06, 25.0, 0.6, 0.004), + Status(carbon_biomass=1.0)), + "Leaf" => ( + MultiScaleModel( + model=ToyAssimModel(), + mapped_variables=[:soil_water_content => "Soil", :aPPFD => "Plant"],), + MultiScaleModel( + model=ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), + mapped_variables=[:TT => "Scene",],), + ToyMaintenanceRespirationModel(2.1, 0.06, 25.0, 1.0, 0.025), + Status(carbon_biomass=1.0)), + "Soil" => (ToySoilWaterModel(),),), + ########## + ModelMapping( + "Default" => ( + Process1Model(1.0), + Status(var1=15.0, var2=0.3,),),), + ########## + ModelMapping( + "Plant" => ( + MultiScaleModel( + model=ToyCAllocationModel(), + mapped_variables=[ + # inputs + :carbon_assimilation => ["Leaf"], + :carbon_demand => ["Leaf", "Internode"], + # outputs + :carbon_allocation => ["Leaf", "Internode"]],), + MultiScaleModel( + model=ToyPlantRmModel(), + mapped_variables=[:Rm_organs => ["Leaf" => :Rm, "Internode" => :Rm],],),), + "Internode" => ( + ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), + ToyMaintenanceRespirationModel(1.5, 0.06, 25.0, 0.6, 0.004), + Status(TT=10.0, carbon_biomass=1.0)), + "Leaf" => ( + MultiScaleModel( + model=ToyAssimModel(), + mapped_variables=[:soil_water_content => "Soil",], + # Notice we provide "Soil", not ["Soil"], so a single value is expected here + ), + ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), + Status(aPPFD=1300.0, TT=10.0, carbon_biomass=1.0), + ToyMaintenanceRespirationModel(2.1, 0.06, 25.0, 1.0, 0.025),), + "Soil" => (ToySoilWaterModel(),),), + ################## ] -out_vars_vectors = [ - [nothing, - NamedTuple(), - Dict(), - #Dict("Leaf" => NamedTuple()), # incorrect - Dict("Leaf" => (:carbon_allocation,),), - Dict("Leaf" => (:carbon_demand,),), - Dict( - "Leaf" => (:carbon_assimilation, :carbon_demand, :soil_water_content, :carbon_allocation), - "Internode" => (:carbon_allocation, :TT_cu_emergence), - "Plant" => (:carbon_allocation,), - "Soil" => (:soil_water_content,),),], - ############# - [nothing, - NamedTuple(), - Dict("Default" => (:var1,)) - ], - ############# - [ - nothing, - NamedTuple(), - Dict( - "Leaf" => (:carbon_assimilation, :carbon_demand), - "Soil" => (:soil_water_content,), - ),], -] + out_vars_vectors = [ + [nothing, + NamedTuple(), + Dict(), + #Dict("Leaf" => NamedTuple()), # incorrect + Dict("Leaf" => (:carbon_allocation,),), + Dict("Leaf" => (:carbon_demand,),), + Dict( + "Leaf" => (:carbon_assimilation, :carbon_demand, :soil_water_content, :carbon_allocation), + "Internode" => (:carbon_allocation, :TT_cu_emergence), + "Plant" => (:carbon_allocation,), + "Soil" => (:soil_water_content,),),], + ############# + [nothing, + NamedTuple(), + Dict("Default" => (:var1,)) + ], + ############# + [ + nothing, + NamedTuple(), + Dict( + "Leaf" => (:carbon_assimilation, :carbon_demand), + "Soil" => (:soil_water_content,), + ),], + ] mtgs = [ - import_mtg_example(), - MultiScaleTreeGraph.Node(MultiScaleTreeGraph.NodeMTG("/", "Default", 0, 0 ),), - import_mtg_example() + import_mtg_example(), + MultiScaleTreeGraph.Node(MultiScaleTreeGraph.NodeMTG("/", "Default", 0, 0),), + import_mtg_example() ] return mtgs, mappings, out_vars_vectors end @@ -323,37 +367,37 @@ end # Split into two parts to ensure eval() syncs and that automatic generation becomes visible for later simulation # See world-age problems and comments around modellist_to_mapping if you don't know/remember what that's about -function test_filtered_output_begin(m::ModelList, status_tuple, requested_outputs, meteo) +function test_filtered_output_begin(m::ModelMapping, status_tuple, requested_outputs, meteo) nsteps = PlantSimEngine.get_nsteps(meteo) preallocated_outputs = PlantSimEngine.pre_allocate_outputs(m, requested_outputs, nsteps) @test length(preallocated_outputs) == nsteps if length(requested_outputs) > 0 @test length(preallocated_outputs[1]) == length(requested_outputs) - else + else # don't compare with the status because unnecessary variables in the status are discarded in the filtered outputs out_vars_all = merge(init_variables(m; verbose=false)...) println(out_vars_all) @test length(preallocated_outputs[1]) == length(out_vars_all) end - - filtered_outputs_modellist = run!(m, meteo; tracked_outputs=requested_outputs, executor = SequentialEx()) + + filtered_outputs_modellist = run!(m, meteo; tracked_outputs=requested_outputs, executor=SequentialEx()) # compare filtered output of a modellist with the filtered output of the equivalent simulation in multiscale mode mtg, mapping, outputs_mapping = PlantSimEngine.modellist_to_mapping(m, status_tuple; nsteps=nsteps, outputs=requested_outputs) - + return mtg, mapping, outputs_mapping, nsteps, filtered_outputs_modellist end function test_filtered_output(mtg, mapping, nsteps, outputs_mapping, meteo, filtered_outputs_modellist) graphsim = PlantSimEngine.GraphSimulation(mtg, mapping, nsteps=nsteps, check=true, outputs=outputs_mapping) - + sim2 = run!(graphsim, - meteo, - PlantMeteo.Constants(), - nothing; - check=true, - executor=SequentialEx() - ) + meteo, + PlantMeteo.Constants(), + nothing; + check=true, + executor=SequentialEx() + ) return compare_outputs_modellist_mapping(filtered_outputs_modellist, graphsim) -end \ No newline at end of file +end diff --git a/test/runtests.jl b/test/runtests.jl index 7458987b1..99cc88cdb 100644 --- a/test/runtests.jl +++ b/test/runtests.jl @@ -18,11 +18,11 @@ include("helper-functions.jl") Aqua.test_all(PlantSimEngine, ambiguities=false) Aqua.test_ambiguities([PlantSimEngine]) - @testset "ModelList" begin - include("test-ModelList.jl") + @testset "ModelMapping: single scale" begin + include("test-ModelMapping.jl") end - @testset "ModelList" begin + @testset "ModelMapping: multi scale" begin include("test-mapping.jl") end diff --git a/test/test-ModelList.jl b/test/test-ModelMapping.jl similarity index 91% rename from test/test-ModelList.jl rename to test/test-ModelMapping.jl index be62807b6..fd70e4cd0 100644 --- a/test/test-ModelList.jl +++ b/test/test-ModelMapping.jl @@ -1,8 +1,8 @@ # Tests: # Defining a list of models without status: -@testset "ModelList with no status" begin - leaf = ModelList( +@testset "ModelMapping with no status" begin + leaf = ModelMapping( process1=Process1Model(1.0), process2=Process2Model() ) @@ -15,7 +15,7 @@ @test length(status(leaf)) == 5 # Requiring 3 steps for initialization: - leaf = ModelList( + leaf = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), ) @@ -45,13 +45,13 @@ end; @test [(process(i), i) for i in models_named] == [(process(i), i) for i in models] end -@testset "ModelList with no process names" begin - with_names = ModelList( +@testset "ModelMapping with no process names" begin + with_names = ModelMapping( process1=Process1Model(1.0), process2=Process2Model() ) - without_names = ModelList( + without_names = ModelMapping( Process1Model(1.0), Process2Model() ) @@ -61,8 +61,8 @@ end @test with_names.status.var2 == without_names.status.var2 end; -@testset "ModelList with a partially initialized status" begin - leaf = ModelList( +@testset "ModelMapping with a partially initialized status" begin + leaf = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), status=(var1=15.0,) @@ -76,7 +76,7 @@ end; @test to_initialize(leaf) == (process1=(:var2,),) # Requiring 3 steps for initialization: - leaf = ModelList( + leaf = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), status=(var1=15.0,), @@ -86,9 +86,9 @@ end; @test status(leaf, :var1) == 15.0 end; -@testset "ModelList with fully initialized status" begin +@testset "ModelMapping with fully initialized status" begin vals = (var1=15.0, var2=0.3) - leaf = ModelList( + leaf = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), status=vals @@ -107,9 +107,9 @@ end; end; -@testset "ModelList with independant models (and missing one in the middle)" begin +@testset "ModelMapping with independant models (and missing one in the middle)" begin vals = (var1=15.0, var2=0.3) - leaf = ModelList( + leaf = ModelMapping( process1=Process1Model(1.0), process3=Process3Model(), status=vals @@ -124,10 +124,10 @@ end; @test [getfield(inits.process3, i) for i in sorted_vars] == fill(-Inf, 3) end; -@testset "Copy a ModelList" begin +@testset "Copy a ModelMapping" begin vars = (var1=15.0, var2=0.3) # Create a model list: - models = ModelList( + models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -154,7 +154,7 @@ end; @test cp_models == Dict("models" => models, "ml3" => ml3) end; -@testset "Convert ModelList status variables into new types" begin +@testset "Convert ModelMapping status variables into new types" begin ref_vars = init_variables( process1=Process1Model(1.0), process2=Process2Model(), @@ -171,9 +171,9 @@ end; end -@testset "ModelList dependencies" begin +@testset "ModelMapping dependencies" begin - models = ModelList( + models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -225,10 +225,10 @@ end=# -@testset "ModelList outputs preallocation" begin +@testset "ModelMapping outputs preallocation" begin meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) vals = (var1=15.0, var2=0.3, TT_cu=cumsum(meteo_day.TT)) - leaf = ModelList( + leaf = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), status=vals @@ -311,6 +311,6 @@ function PlantSimEngine.outputs_(::Reeb) (LAI=-Inf,) end -@testset "ModelList simple cyclic dependency detection" begin - @test_throws "Cyclic" m = ModelList(Beer(0.5), Reeb(0.5)) +@testset "ModelMapping simple cyclic dependency detection" begin + @test_throws "Cyclic" m = ModelMapping(Beer(0.5), Reeb(0.5)) end \ No newline at end of file diff --git a/test/test-Status.jl b/test/test-Status.jl index a121d4493..0bdcfac35 100644 --- a/test/test-Status.jl +++ b/test/test-Status.jl @@ -30,29 +30,29 @@ @test mnt2.b == "hello" end -@testset "Testing ModelList Status" begin - # Create a ModelList - models = ModelList( +@testset "Testing ModelMapping Status" begin + # Create a ModelMapping + models = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), status=(var1=[15.0, 16.0], var2=0.3) ) - @test typeof(status(models)) == Status{ - (:var5, :var4, :var6, :var1, :var3, :var2), - Tuple{ - Base.RefValue{Float64}, Base.RefValue{Float64}, Base.RefValue{Float64}, - Base.RefValue{Vector{Float64}}, Base.RefValue{Float64}, Base.RefValue{Float64}} - } - - + @test typeof(status(models)) == Status{ + (:var5, :var4, :var6, :var1, :var3, :var2), + Tuple{ + Base.RefValue{Float64},Base.RefValue{Float64},Base.RefValue{Float64}, + Base.RefValue{Vector{Float64}},Base.RefValue{Float64},Base.RefValue{Float64}} + } + + @test status(models) == models.status @test status(models)[1] == status(models, 1) - @test typeof(status(models, 1)) == Float64 - @test typeof(status(models, 4)) == Vector{Float64} - + @test typeof(status(models, 1)) == Float64 + @test typeof(status(models, 4)) == Vector{Float64} + @test status(models, :var1)[1] == 15.0 @test status(models, 6) == 0.3 @test status(models, :var1) == [15.0, 16.0] @@ -68,8 +68,8 @@ end #@test models[:var6] = [5.5, 5.8] #@test status(models, :var6) == [5.5, 5.8] - # Testing a vector of ModelList: + # Testing a vector of ModelMapping: @test status([models, models]) == [models.status, models.status] - # Testing a Dict of ModelList: + # Testing a Dict of ModelMapping: @test status(Dict(:m1 => models, :m2 => models)) == Dict(:m1 => models.status, :m2 => models.status) end \ No newline at end of file diff --git a/test/test-corner-cases.jl b/test/test-corner-cases.jl index e1c9ac48e..d568e6b7d 100644 --- a/test/test-corner-cases.jl +++ b/test/test-corner-cases.jl @@ -172,7 +172,7 @@ end @testset "Multiscale nested hard dependencies" begin - mapping3Lvl = Dict("E1" => ( + mapping3Lvl = ModelMapping("E1" => ( Msg3LvlScaleAmontModel(), MultiScaleModel( model=Msg3LvlScaleAvalModel(), @@ -332,7 +332,7 @@ end # actual testset @testset "Soft dependency whose parent is a hard dependency of a parent at a different scale" begin - mapping = Dict( + mapping = ModelMapping( "E1" => (HardDepSameScaleEchelle1Model(), MultiScaleModel( model=HardDepSameScaleEchelle1bisModel(), @@ -449,7 +449,7 @@ end @testset "Process/model reuse at different scales" begin - mapping = Dict( + mapping = ModelMapping( "E1" => ( SingleModelScale1(), Status(in=1.0, in1=1.0), @@ -518,7 +518,7 @@ end using PlantSimEngine, PlantMeteo, DataFrames using PlantSimEngine.Examples mtg = import_mtg_example() - m = Dict( + m = ModelMapping( "Leaf" => ( Process1Model(1.0), Status(var1=10.0, var2=1.0,) @@ -554,7 +554,7 @@ end outs = Dict("Default" => (:var1,)) mtg = MultiScaleTreeGraph.Node(MultiScaleTreeGraph.NodeMTG("/", "Default", 0, 0),) - mapping = Dict( + mapping = ModelMapping( "Default" => ( Process1Model(1.0), Status(var1=15.0, var2=0.3,), @@ -581,7 +581,7 @@ Inputs : a, b, c Outputs : d, e """ -struct ToyToyModel{T} <: AbstractToyModel +struct ToyToyModel{T} <: AbstractToyModel internal_constant::T end @@ -605,13 +605,13 @@ end Atmosphere(T=18.0, Wind=1.0, Rh=0.65, Ri_PAR_f=100.0), ]) - model = ModelList( + model = ModelMapping( ToyToyModel(1), status=(a=1, b=0, c=0), #nsteps = length(meteo) - ) + ) @test to_initialize(model) == NamedTuple() sim = run!(model, meteo) @test DataFrames.nrow(sim) == PlantSimEngine.get_nsteps(meteo) -end \ No newline at end of file +end diff --git a/test/test-dimensions.jl b/test/test-dimensions.jl index 26b27d538..01eeae5a7 100644 --- a/test/test-dimensions.jl +++ b/test/test-dimensions.jl @@ -30,15 +30,15 @@ # @test_throws DimensionMismatch PlantSimEngine.check_dimensions(tst2, w1) # @test_throws DimensionMismatch PlantSimEngine.check_dimensions(tst3, w2) - # ModelList and Weather must be checked for equal length - m1 = ModelList( + # ModelMapping and Weather must be checked for equal length + m1 = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), status=(var1=1.0, var2=2.0) ) @test PlantSimEngine.check_dimensions(m1, w1) === nothing - m2 = ModelList( + m2 = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), status=(var1=[1.0, 2.0], var2=2.0) diff --git a/test/test-fitting.jl b/test/test-fitting.jl index dfb685c3b..54a0337a4 100644 --- a/test/test-fitting.jl +++ b/test/test-fitting.jl @@ -3,7 +3,7 @@ @testset "Fitting Beer" begin k = 0.6 meteo = Atmosphere(T=20.0, Wind=1.0, P=101.3, Rh=0.65, Ri_PAR_f=300.0) - m = ModelList(Beer(k), status=(LAI=2.0,)) + m = ModelMapping(Beer(k), status=(LAI=2.0,)) outs = run!(m, meteo) df = DataFrame(aPPFD=outs[:aPPFD][1], LAI=m.status.LAI[1], Ri_PAR_f=meteo.Ri_PAR_f[1]) diff --git a/test/test-mapping.jl b/test/test-mapping.jl index 96dda7c33..0ed6e9b93 100755 --- a/test/test-mapping.jl +++ b/test/test-mapping.jl @@ -1,4 +1,4 @@ -mapping = Dict( +mapping = ModelMapping( "Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), @@ -54,22 +54,180 @@ for (proc, node) in dep_graph.roots # proc = ("Soil" => :soil_water) ; node = de @test root_models[proc] == node.value end +@testset "ModelMapping checks and normalization" begin + mapping_struct = PlantSimEngine.ModelMapping(Dict(mapping)) + @test mapping_struct isa PlantSimEngine.ModelMapping + @test Set(keys(mapping_struct)) == Set(keys(mapping)) + @test hasmethod(PlantSimEngine.dep, Tuple{PlantSimEngine.ModelMapping}) + @test hasmethod(PlantSimEngine.hard_dependencies, Tuple{PlantSimEngine.ModelMapping}) + @test hasmethod(PlantSimEngine.inputs, Tuple{PlantSimEngine.ModelMapping}) + @test hasmethod(PlantSimEngine.outputs, Tuple{PlantSimEngine.ModelMapping}) + @test hasmethod(PlantSimEngine.variables, Tuple{PlantSimEngine.ModelMapping}) + @test hasmethod(PlantSimEngine.to_initialize, Tuple{PlantSimEngine.ModelMapping}) + @test hasmethod(PlantSimEngine.reverse_mapping, Tuple{PlantSimEngine.ModelMapping}) + + mapping_from_pairs = PlantSimEngine.ModelMapping( + "Plant" => mapping["Plant"], + "Internode" => mapping["Internode"], + "Leaf" => mapping["Leaf"], + "Soil" => mapping["Soil"], + ) + @test Set(keys(mapping_from_pairs)) == Set(keys(mapping)) + + mapping_with_specs = PlantSimEngine.ModelMapping( + "Scene" => (ModelSpec(ToyDegreeDaysCumulModel()) |> TimeStepModel(ClockSpec(24.0, 1.0)),), + "Soil" => (ModelSpec(ToySoilWaterModel()) |> TimeStepModel(ClockSpec(24.0, 1.0)),), + "Leaf" => ( + ModelSpec(ToyAssimModel()) |> + MultiScaleModel([:soil_water_content => "Soil"]) |> + TimeStepModel(1.0), + ), + ) + @test mapping_with_specs isa PlantSimEngine.ModelMapping + @test any(item -> item isa ModelSpec, mapping_with_specs["Soil"]) + + dep_from_dict = dep(mapping) + dep_from_struct = dep(mapping_struct) + @test Set(keys(dep_from_dict.roots)) == Set(keys(dep_from_struct.roots)) + @test Set(keys(first(PlantSimEngine.hard_dependencies(mapping_struct)).roots)) == Set(keys(first(PlantSimEngine.hard_dependencies(Dict(mapping_struct))).roots)) + @test inputs(mapping_struct) == inputs(Dict(mapping_struct)) + @test outputs(mapping_struct) == outputs(Dict(mapping_struct)) + @test variables(mapping_struct) == variables(Dict(mapping_struct)) + + ModelMapping_scale = ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + status=(var1=1.0, var2=2.0) + ) + merged_mapping = PlantSimEngine.ModelMapping(Dict("Default" => ModelMapping_scale)) + @test length(PlantSimEngine.get_models(merged_mapping["Default"])) == 2 + @test !isnothing(PlantSimEngine.get_status(merged_mapping["Default"])) + + single_scale_from_models = PlantSimEngine.ModelMapping( + Process1Model(1.0), + Process2Model(); + scale="Default", + status=(var1=1.0, var2=2.0), + ) + @test Set(keys(single_scale_from_models)) == Set(["Default"]) + @test length(PlantSimEngine.get_models(single_scale_from_models["Default"])) == 2 + @test PlantSimEngine.get_status(single_scale_from_models["Default"]).var1 == 1.0 + + single_scale_from_namedtuple = PlantSimEngine.ModelMapping( + (process1=Process1Model(1.0), process2=Process2Model()); + status=(var1=1.0, var2=2.0), + ) + @test Set(keys(single_scale_from_namedtuple)) == Set(["Default"]) + @test length(PlantSimEngine.get_models(single_scale_from_namedtuple["Default"])) == 2 + + single_scale_from_kwargs = PlantSimEngine.ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + status=(var1=1.0, var2=2.0), + ) + @test Set(keys(single_scale_from_kwargs)) == Set(["Default"]) + @test length(PlantSimEngine.get_models(single_scale_from_kwargs["Default"])) == 2 + + @test_throws "Cannot mix scale-level pairs" PlantSimEngine.ModelMapping( + "Leaf" => (Process1Model(1.0),), + process2=Process2Model(), + ) + + missing_scale_mapping = Dict( + "Leaf" => ( + MultiScaleModel( + model=ToyAssimModel(), + mapped_variables=[:soil_water_content => "Soil"], + ), + ), + ) + @test_throws "missing scale `Soil`" PlantSimEngine.ModelMapping(missing_scale_mapping) + + missing_source_variable_mapping = Dict( + "Leaf" => ( + MultiScaleModel( + model=ToyAssimModel(), + mapped_variables=[:soil_water_content => "Soil"], + ), + ), + "Soil" => ( + Process1Model(1.0), + ), + ) + @test_throws "not available at scale `Soil`" PlantSimEngine.ModelMapping(missing_source_variable_mapping) + + no_model_mapping = Dict( + "Soil" => (Status(soil_water_content=0.2),), + ) + @test_throws "defines no model" PlantSimEngine.ModelMapping(no_model_mapping) + + duplicate_process_mapping = Dict( + "Leaf" => ( + Process1Model(1.0), + Process1Model(2.0), + ), + ) + @test_throws "duplicate process(es)" PlantSimEngine.ModelMapping(duplicate_process_mapping) + + meteo = Atmosphere(T=20.0, Wind=1.0, Rh=0.65) + models_single_scale = ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + status=(var1=15.0, var2=0.3) + ) + baseline_outputs = run!(models_single_scale, meteo) + + outputs_from_models_args = run!( + PlantSimEngine.ModelMapping( + Process1Model(1.0), + Process2Model(), + Process3Model(); + status=(var1=15.0, var2=0.3), + ), + meteo + ) + @test outputs_from_models_args == baseline_outputs + + outputs_from_named_tuple = run!( + PlantSimEngine.ModelMapping( + (process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model()); + status=(var1=15.0, var2=0.3), + ), + meteo + ) + @test outputs_from_named_tuple == baseline_outputs + + outputs_from_kwargs = run!( + PlantSimEngine.ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + status=(var1=15.0, var2=0.3), + ), + meteo + ) + @test outputs_from_kwargs == baseline_outputs + + @test_throws "Use `run!(mtg, mapping, ...)` for multiscale mappings." run!(mapping, meteo) +end + ########################### -### ModelList vs Mapping comparison +### Single and multi-scale ModelMapping comparison ### and Mapping with custom models vs mapping with generated models for user-provided vector ########################### # Currently untested in 'real' multi-scale modes, or with complex configs (hard dependencies). # Need to place the simple timestep models in PlantSimEngine, and probably provide more complex ones at some point -# And then need to insert it at the graph sim generation level, and modify tests to consistently do modellist <-> mapping conversions +# And then need to insert it at the graph sim generation level, and modify tests to consistently do single <-> multiple scale conversions # And then implement tests with proper output filtering @testset "check_statuses_contain_no_remaining_vectors behaviour" begin meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) - mapping_with_vector = Dict( + mapping_with_vector = ModelMapping( "Scale" => (ToyAssimGrowthModel(0.0, 0.0, 0.0), ToyCAllocationModel(), @@ -81,7 +239,7 @@ end @test !last(PlantSimEngine.check_statuses_contain_no_remaining_vectors(mapping_with_vector)) @test_throws "call the function generate_models_from_status_vectors" PlantSimEngine.GraphSimulation(mtg, mapping_with_vector) - mapping_with_empty_status = Dict( + mapping_with_empty_status = ModelMapping( "Scale" => (ToyAssimGrowthModel(0.0, 0.0, 0.0), ToyCAllocationModel(), @@ -92,140 +250,13 @@ end @test last(PlantSimEngine.check_statuses_contain_no_remaining_vectors(mapping_with_empty_status)) end -# simple conversion to a mapping, with a manually written model -function modellist_to_mapping_manual(modellist_original::ModelList, modellist_status, nsteps; check=true, outputs=nothing, TT_cu_vec=Vector{Float64}()) - - modellist = Base.copy(modellist_original, modellist_original.status) - - default_scale = "Default" - mtg = MultiScaleTreeGraph.Node(MultiScaleTreeGraph.NodeMTG("/", default_scale, 0, 0),) - - #models = collect(values(object)) - models = modellist.models - #status_ts = modellist.status.ts - - mapping = Dict( - default_scale => ( - models..., - ToyTestDegreeDaysCumulModel(TT_cu_vec=TT_cu_vec), - PlantSimEngine.HelperNextTimestepModel(), - MultiScaleModel( - model=PlantSimEngine.HelperCurrentTimestepModel(), - mapped_variables=[PreviousTimeStep(:next_timestep),], - ), - Status(current_timestep=1, next_timestep=1) - ), - ) - - if isnothing(outputs) - f = [] - for i in 1:length(modellist.models) - aa = init_variables(modellist.models[i]) - bb = keys(aa) - for j in 1:length(bb) - push!(f, bb[j]) - end - #f = (f..., bb...) - end - - f = unique!(f) - all_vars = (f...,) - #all_vars = merge((keys(init_variables(object.models[i])) for i in 1:length(object.models))...) - else - all_vars = outputs - # TODO sanity check - end - - sim = PlantSimEngine.GraphSimulation(mtg, mapping, nsteps=nsteps, check=check, outputs=Dict(default_scale => all_vars)) - return sim -end - - -PlantSimEngine.@process "Degreedays" verbose = false - -struct ToyTestDegreeDaysCumulModel <: AbstractDegreedaysModel - TT_cu_vec::Vector{Float64} -end - -PlantSimEngine.inputs_(::ToyTestDegreeDaysCumulModel) = (current_timestep=1,) -PlantSimEngine.outputs_(::ToyTestDegreeDaysCumulModel) = (TT_cu=0.0,) - -ToyTestDegreeDaysCumulModel(; TT_cu_vec=Vector{Float64}()) = ToyTestDegreeDaysCumulModel(TT_cu_vec) - - -function PlantSimEngine.run!(m::ToyTestDegreeDaysCumulModel, models, status, meteo, constants=nothing, extra=nothing) - status.TT_cu = m.TT_cu_vec[status.current_timestep] -end - -PlantSimEngine.ObjectDependencyTrait(::Type{<:ToyTestDegreeDaysCumulModel}) = PlantSimEngine.IsObjectDependent() - -@testset "ModelList and Mapping result consistency" begin - - meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) - - st = (TT_cu=cumsum(meteo_day.TT),) - - TT_cu_vec = Vector(cumsum(meteo_day.TT)) - - rue = 0.3 - models = ModelList( - ToyLAIModel(), - Beer(0.5), - ToyRUEGrowthModel(rue), - status=st, - ) - - modellist_outputs = run!(models, - meteo_day - ; - check=true, - executor=ThreadedEx() - ) - - nsteps = nrow(meteo_day) - graphsim = modellist_to_mapping_manual(models, st, nsteps; outputs=nothing, TT_cu_vec=TT_cu_vec) - - sim = run!(graphsim, - meteo_day, - PlantMeteo.Constants(), - nothing; - check=true, - executor=SequentialEx() - ) - - @test compare_outputs_modellist_mapping(modellist_outputs, graphsim) - - # fully automated model generation - st2 = (TT_cu=Vector(cumsum(meteo_day.TT)),) - - mtg, mapping, outputs_mapping = PlantSimEngine.modellist_to_mapping(models, st2; nsteps=nsteps, outputs=nothing) - - @test to_initialize(mapping) == Dict() - - graphsim2 = PlantSimEngine.GraphSimulation(mtg, mapping, nsteps=nsteps, check=true, outputs=outputs_mapping) - - sim2 = run!(graphsim2, - meteo_day, - PlantMeteo.Constants(), - nothing; - check=true, - executor=SequentialEx() - ) - @test compare_outputs_modellist_mapping(modellist_outputs, graphsim2) - @test compare_outputs_graphsim(graphsim, graphsim2) - -end - -#[getproperty(a,i) for i in fieldnames(typeof(a))] - - @testset "Vector in status in a multiscale context" begin meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), DataFrame, header=18) TT_v = Vector(meteo_day.TT) TT_cu_vec = Vector(cumsum(meteo_day.TT)) nsteps = length(meteo_day.TT) - mapping_with_vector = Dict("Plant" => ( + mapping_with_vector = ModelMapping("Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), mapped_variables=[ @@ -283,7 +314,7 @@ end for i in nsteps carbon_biomass_vec[i] = 2.0 end - mapping_with_two_vectors = Dict("Plant" => ( + mapping_with_two_vectors = ModelMapping("Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), mapped_variables=[ @@ -333,4 +364,4 @@ end ) @test compare_outputs_graphsim(graph_sim_multiscale, graph_sim_multiscale_2) -end \ No newline at end of file +end diff --git a/test/test-mtg-dynamic.jl b/test/test-mtg-dynamic.jl index 164e75b5a..952533133 100644 --- a/test/test-mtg-dynamic.jl +++ b/test/test-mtg-dynamic.jl @@ -10,7 +10,7 @@ meteo = Weather( ] ) -mapping = Dict( +mapping = ModelMapping( "Scene" => ToyDegreeDaysCumulModel(), "Plant" => ( MultiScaleModel( diff --git a/test/test-mtg-multiscale-cyclic-dep.jl b/test/test-mtg-multiscale-cyclic-dep.jl index ddb79492c..662673b9b 100644 --- a/test/test-mtg-multiscale-cyclic-dep.jl +++ b/test/test-mtg-multiscale-cyclic-dep.jl @@ -15,7 +15,7 @@ out_vars = Dict( ) @testset "Cyclic dependency -> error" begin - mapping_cyclic = Dict( + mapping_cyclic = ModelMapping( "Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), @@ -61,7 +61,7 @@ end @testset "Cyclic dependency -> fixed with `PreviousTimeStep`" begin - mapping_nocyclic = Dict( + mapping_nocyclic = ModelMapping( "Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), @@ -123,7 +123,7 @@ end end @testset "Mutiscale simulation -> cyclic dependency" begin - mapping = Dict( + mapping = ModelMapping( "Scene" => ( ToyDegreeDaysCumulModel(), MultiScaleModel( diff --git a/test/test-mtg-multiscale.jl b/test/test-mtg-multiscale.jl index b0df935a3..8bc5e2375 100644 --- a/test/test-mtg-multiscale.jl +++ b/test/test-mtg-multiscale.jl @@ -14,7 +14,7 @@ meteo = Weather( var2 = 0.3 leaf[:var2] = var2 - mapping = Dict( + mapping = ModelMapping( "Leaf" => ( Process1Model(1.0), Process2Model(), @@ -31,7 +31,7 @@ meteo = Weather( end # A mapping that actually works (same as before but with the init for TT): -mapping_1 = Dict( +mapping_1 = ModelMapping( "Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), @@ -225,7 +225,7 @@ end # Testing the mappings: @testset "Mapping: missing initialisation" begin - mapping = Dict( + mapping = ModelMapping( "Plant" => MultiScaleModel( model=ToyCAllocationModel(), @@ -266,7 +266,7 @@ end end @testset "Mapping: missing organ in mapping (Soil)" begin - mapping = Dict( + @test_throws "missing scale `Soil`" ModelMapping( "Plant" => MultiScaleModel( model=ToyCAllocationModel(), @@ -289,12 +289,6 @@ end Status(aPPFD=1300.0, TT=10.0), ) ) - - if VERSION < v"1.8" # We test differently depending on the julia version because the format of the error message changed - @test_throws AssertionError to_initialize(mapping) - else - @test_throws "Scale Soil not found in the mapping, but mapped to the Leaf scale." to_initialize(mapping) - end end mtg_var = let @@ -307,7 +301,7 @@ mtg_var = let end @testset "Mapping: missing model at other scale (soil_water_content) + missing init + var1 from MTG" begin - mapping = Dict( + @test_throws "not available at scale `Soil`" ModelMapping( "Plant" => MultiScaleModel( model=ToyCAllocationModel(), @@ -333,12 +327,10 @@ end Process1Model(1.0), ), ) - - @test_throws "The variable `soil_water_content` is mapped from scale `Soil` to scale `Leaf`, but is not computed by any model at `Soil` scale." to_initialize(mapping, mtg_var) end @testset "Mapping: missing init + var1 from MTG" begin - mapping = Dict( + mapping = ModelMapping( "Plant" => MultiScaleModel( model=ToyCAllocationModel(), @@ -380,7 +372,7 @@ end @testset "run! on MTG: simple mapping" begin #out = @test_nowarn run!(mtg, Dict("Soil" => (ToySoilWaterModel(),)), meteo) nsteps = PlantSimEngine.get_nsteps(meteo) - sim = PlantSimEngine.GraphSimulation(mtg, Dict("Soil" => (ToySoilWaterModel(),)), nsteps=nsteps, check=true, outputs=nothing) + sim = PlantSimEngine.GraphSimulation(mtg, ModelMapping("Soil" => (ToySoilWaterModel(),)), nsteps=nsteps, check=true, outputs=nothing) out = run!(sim,meteo) @test sim.statuses["Soil"][1].node == soil @@ -390,7 +382,7 @@ end @test collect(keys(sim.dependency_graph.roots))[1] == Pair("Soil", :soil_water) @test sim.graph == mtg - leaf_mapping = Dict("Leaf" => (ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), Status(TT=10.0))) + leaf_mapping = ModelMapping("Leaf" => (ToyCDemandModel(optimal_biomass=10.0, development_duration=200.0), Status(TT=10.0))) #out = run!(mtg, leaf_mapping, meteo) nsteps = PlantSimEngine.get_nsteps(meteo) @@ -408,7 +400,7 @@ end # A mapping with all different types of mapping (single, multi-scale, model as is, or tuple of): @testset "run! on MTG with complete mapping (missing init)" begin - mapping_all = Dict( + mapping_all = ModelMapping( "Plant" => MultiScaleModel( model=ToyCAllocationModel(), @@ -537,7 +529,7 @@ end # Here we initialise var1 to a constant value: @testset "MTG initialisation" begin var1 = 1.0 - mapping = Dict( + mapping = ModelMapping( "Leaf" => ( Process1Model(1.0), Process2Model(), @@ -552,7 +544,7 @@ end @test_throws "Variable `var2` is not computed by any model, not initialised by the user in the status, and not found in the MTG at scale Leaf (checked for MTG node 5)." PlantSimEngine.init_simulation(mtg, mapping) end - mapping = Dict( + mapping = ModelMapping( "Leaf" => ( Process1Model(1.0), Process2Model(), @@ -573,8 +565,7 @@ end end @testset "MTG with complex mapping" begin - mapping = - Dict( + mapping = ModelMapping( "Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), @@ -631,8 +622,7 @@ end end @testset "MTG with dynamic output variables" begin - mapping = - Dict( + mapping = ModelMapping( "Plant" => ( MultiScaleModel( model=ToyCAllocationModel(), @@ -723,4 +713,4 @@ end #A2 = outputs(out, 5) #@test A == A2 -end \ No newline at end of file +end diff --git a/test/test-multirate-output-export.jl b/test/test-multirate-output-export.jl index 84fb81ebc..984af5ae3 100644 --- a/test/test-multirate-output-export.jl +++ b/test/test-multirate-output-export.jl @@ -53,7 +53,7 @@ PlantSimEngine.timespec(::Type{<:MRDefaultSceneAggModel}) = ClockSpec(4.0, 1.0) meteo4 = Weather(repeat([Atmosphere(T=20.0, Wind=1.0, Rh=0.65)], 4)) # Stream-only producer remains exportable when process is explicit. - mapping_stream = Dict( + mapping_stream = ModelMapping( "Leaf" => ( ModelSpec(MRExportSourceModel(Ref(0))) |> TimeStepModel(1.0) |> @@ -89,7 +89,7 @@ PlantSimEngine.timespec(::Type{<:MRDefaultSceneAggModel}) = ClockSpec(4.0, 1.0) ) # Canonical routing allows omitting process in requests. - mapping_canonical = Dict( + mapping_canonical = ModelMapping( "Leaf" => ( ModelSpec(MRExportSourceModel(Ref(0))) |> TimeStepModel(1.0), @@ -110,7 +110,7 @@ PlantSimEngine.timespec(::Type{<:MRDefaultSceneAggModel}) = ClockSpec(4.0, 1.0) # Optional direct export return from run! on GraphSimulation. sim_direct = PlantSimEngine.GraphSimulation( mtg, - Dict( + ModelMapping( "Leaf" => ( ModelSpec(MRExportSourceModel(Ref(0))) |> TimeStepModel(1.0), @@ -158,7 +158,7 @@ end meteo8 = Weather(repeat([Atmosphere(T=20.0, Wind=1.0, Rh=0.65)], 8)) - mapping_defaults = Dict( + mapping_defaults = ModelMapping( "Leaf" => ( MultiScaleModel( model=MRDefaultLeafSourceModel(Ref(0)), diff --git a/test/test-multirate-runtime.jl b/test/test-multirate-runtime.jl index ecf9f3d8d..f9aa60242 100644 --- a/test/test-multirate-runtime.jl +++ b/test/test-multirate-runtime.jl @@ -280,7 +280,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( internode = Node(plant, MultiScaleTreeGraph.NodeMTG("/", "Internode", 1, 2)) Node(internode, MultiScaleTreeGraph.NodeMTG("+", "Leaf", 1, 2)) - mapping_ok = Dict( + mapping_ok = ModelMapping( "Leaf" => ( MRSourceModel(), ModelSpec(MROverwriteModel()) |> OutputRouting(; C=:stream_only), @@ -328,7 +328,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test sim_ok.temporal_state.caches[key_dir].v == 10.0 @test sim_ok.temporal_state.caches[key_auto].v == 10.0 - mapping_conflict = Dict( + mapping_conflict = ModelMapping( "Leaf" => ( MRConflict1Model(), MRConflict2Model(), @@ -340,7 +340,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Expectation 6: models run at different clocks; slower model holds last value between runs. source_counter = Ref(0) - mapping_clock_trait = Dict( + mapping_clock_trait = ModelMapping( "Leaf" => ( ModelSpec(MRClockSourceModel(source_counter)) |> TimeStepModel(1.0), ModelSpec(MRClockConsumerModel()) |> @@ -359,7 +359,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Expectation 7: TimeStepModel override takes precedence over model timespec. source_counter_2 = Ref(0) - mapping_clock_override = Dict( + mapping_clock_override = ModelMapping( "Leaf" => ( ModelSpec(MRClockSourceModel(source_counter_2)) |> TimeStepModel(1.0), ModelSpec(MRClockConsumerModel()) |> @@ -376,7 +376,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test sim_clock_override.temporal_state.last_run[ModelKey(scope, "Leaf", :mrclockconsumer)] == 3.0 # Expectation 7b: non-sequential executors warn and fall back to sequential behavior. - mapping_clock_fallback_seq = Dict( + mapping_clock_fallback_seq = ModelMapping( "Leaf" => ( ModelSpec(MRClockSourceModel(Ref(0))) |> TimeStepModel(1.0), ModelSpec(MRClockConsumerModel()) |> @@ -387,7 +387,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( out_fallback_seq = run!(sim_clock_fallback_seq, meteo4, multirate=true, executor=SequentialEx()) out_fallback_seq_df = convert_outputs(out_fallback_seq, DataFrame) - mapping_clock_fallback_threaded = Dict( + mapping_clock_fallback_threaded = ModelMapping( "Leaf" => ( ModelSpec(MRClockSourceModel(Ref(0))) |> TimeStepModel(1.0), ModelSpec(MRClockConsumerModel()) |> @@ -405,7 +405,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Expectation 8: cross-scale hold-last resolution works with different clocks. # Leaf producer runs each step; Plant consumer runs every 2 steps (1, 3) and reads Leaf XS through multiscale mapping. source_counter_3 = Ref(0) - mapping_cross = Dict( + mapping_cross = ModelMapping( "Leaf" => ( ModelSpec(MRCrossSourceModel(source_counter_3)) |> TimeStepModel(1.0), ), @@ -425,7 +425,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Expectation 8a: cross-scale producer is inferred automatically when unique. source_counter_3_auto = Ref(0) - mapping_cross_auto = Dict( + mapping_cross_auto = ModelMapping( "Leaf" => ( ModelSpec(MRCrossSourceModel(source_counter_3_auto)) |> TimeStepModel(1.0), ), @@ -453,7 +453,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( Node(internode2_b, MultiScaleTreeGraph.NodeMTG("+", "Leaf", 1, 2)) source_counter_scoped = Ref(0) - mapping_scoped = Dict( + mapping_scoped = ModelMapping( "Leaf" => ( ModelSpec(MRCrossSourceModel(source_counter_scoped)) |> TimeStepModel(1.0) |> ScopeModel(:plant), ), @@ -491,7 +491,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Consumer runs every step and receives XI through Interpolate: # expected YI over time is [1, 1, 3, 4, 5]. interp_counter = Ref(0) - mapping_interp = Dict( + mapping_interp = ModelMapping( "Leaf" => ( ModelSpec(MRInterpSourceModel(interp_counter)) |> TimeStepModel(ClockSpec(2.0, 1.0)), ModelSpec(MRInterpConsumerModel()) |> @@ -512,7 +512,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # - at t=3: window [2,3] => mean([2,3]) = 2.5 # Output YA over time is therefore [1, 1, 2.5, 2.5]. agg_counter = Ref(0) - mapping_agg = Dict( + mapping_agg = ModelMapping( "Leaf" => ( ModelSpec(MRAggSourceModel(agg_counter)) |> TimeStepModel(1.0), ModelSpec(MRAggConsumerModel()) |> @@ -535,7 +535,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Source XA=[1,2,3,4], consumer runs at t=1,3 with reducer=MaxReducer(). # YA over time is [1,1,3,3]. agg_counter_max = Ref(0) - mapping_agg_max = Dict( + mapping_agg_max = ModelMapping( "Leaf" => ( ModelSpec(MRAggSourceModel(agg_counter_max)) |> TimeStepModel(1.0), ModelSpec(MRAggConsumerModel()) |> @@ -552,7 +552,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Source XA=[1,2,3,4], consumer runs at t=1,3 with reducer=max-min. # YA over time is [0,0,1,1]. agg_counter_callable = Ref(0) - mapping_integrate_callable = Dict( + mapping_integrate_callable = ModelMapping( "Leaf" => ( ModelSpec(MRAggSourceModel(agg_counter_callable)) |> TimeStepModel(1.0), ModelSpec(MRAggConsumerModel()) |> @@ -569,7 +569,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Source runs at t=1,3,5 with values 1,3,5. Consumer runs each step. # With Interpolate(mode=:hold, extrapolation=:hold), YI is [1,1,3,3,5,5]. interp_counter_hold = Ref(0) - mapping_interp_hold = Dict( + mapping_interp_hold = ModelMapping( "Leaf" => ( ModelSpec(MRInterpSourceModel(interp_counter_hold)) |> TimeStepModel(ClockSpec(2.0, 1.0)), ModelSpec(MRInterpConsumerModel()) |> @@ -587,7 +587,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Source runs at t=1 and t=25 (ClockSpec(24,1)), consumer runs every step. # YD should stay at 1 for t=1..24, then switch to 2 at t=25. daily_counter = Ref(0) - mapping_daily_hourly = Dict( + mapping_daily_hourly = ModelMapping( "Leaf" => ( ModelSpec(MRDailySourceModel(daily_counter)) |> TimeStepModel(ClockSpec(24.0, 1.0)), ModelSpec(MRHourlyFromDailyConsumerModel()) |> @@ -607,7 +607,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( # Expectation 15: period-based timestep uses timeline base-step conversion. # Meteo has duration=Hour(1), source uses Day(1) => runs on t=1 and t=25. daily_counter_period = Ref(0) - mapping_daily_period = Dict( + mapping_daily_period = ModelMapping( "Leaf" => ( ModelSpec(MRDailySourceModel(daily_counter_period)) |> TimeStepModel(Dates.Day(1)), ModelSpec(MRHourlyFromDailyConsumerModel()) |> @@ -624,7 +624,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test sim_daily_period.temporal_state.last_run[ModelKey(scope, "Leaf", :mrdailysource)] == 25.0 # Expectation 16: model timesteps shorter than meteo base step are rejected. - mapping_substep_period = Dict( + mapping_substep_period = ModelMapping( "Leaf" => ( ModelSpec(MRDailySourceModel(Ref(0))) |> TimeStepModel(Dates.Minute(30)), ), @@ -636,7 +636,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( range_counter_a = Ref(0) range_counter_b = Ref(0) range_counter_forced = Ref(0) - mapping_timestep_hints = Dict( + mapping_timestep_hints = ModelMapping( "Leaf" => ( ModelSpec(MRRangeHintAModel(range_counter_a)), ModelSpec(MRRangeHintBModel(range_counter_b)), @@ -662,7 +662,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( if _HAS_METEO_SAMPLER_API # Expectation 18: meteo is sampled at model clock using default weather aggregation. - mapping_meteo_default = Dict( + mapping_meteo_default = ModelMapping( "Leaf" => ( ModelSpec(MRMeteoDailyConsumerModel()) |> TimeStepModel(ClockSpec(2.0, 1.0)), @@ -704,7 +704,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test isapprox(out_meteo_default_mtg_df["Leaf"][:, :MSWq][3], 1.8; atol=1.0e-9) # Expectation 19: meteo bindings allow custom reducers and variable remapping. - mapping_meteo_custom = Dict( + mapping_meteo_custom = ModelMapping( "Leaf" => ( ModelSpec(MRMeteoCustomConsumerModel()) |> TimeStepModel(ClockSpec(2.0, 1.0)) |> @@ -734,7 +734,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( ) for h in 1:24 ]) - mapping_meteo_hint = Dict( + mapping_meteo_hint = ModelMapping( "Leaf" => ( ModelSpec(MRMeteoHintConsumerModel()), ), @@ -778,7 +778,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( ], )) - mapping_meteo_calendar_current = Dict( + mapping_meteo_calendar_current = ModelMapping( "Leaf" => ( ModelSpec(MRMeteoDailyConsumerModel()) |> TimeStepModel(1.0) |> @@ -799,7 +799,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test isapprox(out_meteo_calendar_current_df["Leaf"][25, :MSWq], 17.28; atol=1.0e-9) # Expectation 22: CalendarWindow(:day, :previous_complete_period) uses previous day. - mapping_meteo_calendar_prev = Dict( + mapping_meteo_calendar_prev = ModelMapping( "Leaf" => ( ModelSpec(MRMeteoDailyConsumerModel()) |> TimeStepModel(1.0) |> @@ -814,7 +814,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test out_meteo_calendar_prev_df["Leaf"][30, :MTmax] == 24.0 # Expectation 23: strict previous-complete-period errors when unavailable. - mapping_meteo_calendar_prev_strict = Dict( + mapping_meteo_calendar_prev_strict = ModelMapping( "Leaf" => ( ModelSpec(MRMeteoDailyConsumerModel()) |> TimeStepModel(1.0) |> @@ -826,7 +826,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( end # Expectation 24: ambiguous same-name inferred producer is rejected at initialization. - mapping_ambiguous_infer = Dict( + mapping_ambiguous_infer = ModelMapping( "Leaf" => ( MRConflict1Model(), MRConflict2Model(), @@ -836,7 +836,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test_throws "Ambiguous inferred producer for input `Z`" PlantSimEngine.GraphSimulation(mtg, mapping_ambiguous_infer, nsteps=1, check=true, outputs=Dict("Leaf" => (:ZZ,))) # Expectation 24a: stream-only publishers are ignored by auto input producer inference. - mapping_stream_only_infer = Dict( + mapping_stream_only_infer = ModelMapping( "Leaf" => ( MRConflict1Model(), ModelSpec(MRConflict2Model()) |> OutputRouting(; Z=:stream_only), @@ -850,7 +850,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test input_bindings(spec_stream_only_infer).Z.process == :mrconflict1 # Expectation 24b: cross-scale inference ignores sibling scales not on the same lineage. - mapping_lineage_infer = Dict( + mapping_lineage_infer = ModelMapping( "Plant" => ( MRAncestorSourceModel(), ), @@ -870,7 +870,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test input_bindings(spec_lineage_infer).Z.scale == "Plant" # Expectation 25: missing producer remains allowed; model can rely on initialized/forced inputs. - mapping_missing_input = Dict( + mapping_missing_input = ModelMapping( "Leaf" => ( MRMissingInputConsumerModel(), Status(U=42.0), @@ -881,7 +881,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test status(sim_missing_input)["Leaf"][1].OU == 42.0 # Expectation 26: invalid mapping-level API configuration fails during GraphSimulation init. - mapping_bad_input = Dict( + mapping_bad_input = ModelMapping( "Leaf" => ( MRSourceModel(), ModelSpec(MRConsumerModel()) |> @@ -890,7 +890,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( ) @test_throws "declares binding for input `Z`" PlantSimEngine.GraphSimulation(mtg, mapping_bad_input, nsteps=1, check=true, outputs=Dict("Leaf" => (:B,))) - mapping_bad_process = Dict( + mapping_bad_process = ModelMapping( "Leaf" => ( ModelSpec(MRConsumerModel()) |> InputBindings(; C=(process=:unknown_process, var=:S)), @@ -898,7 +898,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( ) @test_throws "Unknown source process `unknown_process`" PlantSimEngine.GraphSimulation(mtg, mapping_bad_process, nsteps=1, check=true, outputs=Dict("Leaf" => (:B,))) - mapping_bad_routing = Dict( + mapping_bad_routing = ModelMapping( "Leaf" => ( ModelSpec(MRSourceModel()) |> OutputRouting(; Z=:stream_only), @@ -906,7 +906,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( ) @test_throws "declares routing for output `Z`" PlantSimEngine.GraphSimulation(mtg, mapping_bad_routing, nsteps=1, check=true, outputs=Dict("Leaf" => (:S,))) - mapping_bad_interp_mode = Dict( + mapping_bad_interp_mode = ModelMapping( "Leaf" => ( MRSourceModel(), ModelSpec(MRConsumerModel()) |> @@ -917,39 +917,39 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( @test_throws "Unsupported reducer value" Aggregate(:median) - mapping_bad_period = Dict( + mapping_bad_period = ModelMapping( "Leaf" => ( ModelSpec(MRDailySourceModel(Ref(0))) |> TimeStepModel(Dates.Month(1)), ), ) @test_throws "non-fixed periods are not supported" PlantSimEngine.GraphSimulation(mtg, mapping_bad_period, nsteps=1, check=true, outputs=Dict("Leaf" => (:XD,))) - mapping_bad_scope = Dict( + mapping_bad_scope = ModelMapping( "Leaf" => ( ModelSpec(MRSourceModel()) |> ScopeModel(:invalid_scope), ), ) @test_throws "Invalid scope selector" PlantSimEngine.GraphSimulation(mtg, mapping_bad_scope, nsteps=1, check=true, outputs=Dict("Leaf" => (:S,))) - mapping_bad_meteo = Dict( + mapping_bad_meteo = ModelMapping( "Leaf" => ( ModelSpec(MRMeteoCustomConsumerModel()) |> - MeteoBindings(; Ri_SW_f=(source=:Ri_SW_f, badfield=:oops)), + MeteoBindings(; Ri_SW_f=(source=:Ri_SW_f, badfield=:oops)), ), ) @test_throws "unsupported fields" PlantSimEngine.GraphSimulation(mtg, mapping_bad_meteo, nsteps=1, check=true, outputs=Dict("Leaf" => (:MRQ,))) - @test_throws "Unsupported MeteoBindings value" Dict( + @test_throws "Unsupported MeteoBindings value" ModelMapping( "Leaf" => ( ModelSpec(MRMeteoCustomConsumerModel()) |> - MeteoBindings(; Ri_SW_f=:radiation_energy), + MeteoBindings(; Ri_SW_f=:radiation_energy), ), ) - @test_throws "Unsupported MeteoWindow value" Dict( + @test_throws "Unsupported MeteoWindow value" ModelMapping( "Leaf" => ( ModelSpec(MRMeteoCustomConsumerModel()) |> - MeteoWindow("day"), + MeteoWindow("day"), ), ) @@ -960,7 +960,7 @@ PlantSimEngine.meteo_hint(::Type{<:MRMeteoHintConsumerModel}) = ( PlantSimEngine.run!(::MRBadHintModel, models, status, meteo, constants=nothing, extra=nothing) = (status.X = 1.0) PlantSimEngine.timestep_hint(::Type{<:MRBadHintModel}) = "hourly" - mapping_bad_hint = Dict( + mapping_bad_hint = ModelMapping( "Leaf" => ( ModelSpec(MRBadHintModel()), ), diff --git a/test/test-performance.jl b/test/test-performance.jl index cd0bf58c8..462615370 100644 --- a/test/test-performance.jl +++ b/test/test-performance.jl @@ -22,22 +22,22 @@ nrows = nrow(meteo_day) vc = [0 for i in 1:nrows] -models1 = ModelList(process1=ToySleepModel(), status=(a=vc,)) -models2 = ModelList(process1=ToySleepModel(), status=(a=vc,)) +mapping1 = ModelMapping(process1=ToySleepModel(), status=(a=vc,)) +mapping2 = ModelMapping(process1=ToySleepModel(), status=(a=vc,)) @testset begin "Check number of threads" nthr = Threads.nthreads() @test nthr >= 1 - t_seq = @benchmark run!(models1, meteo_day; executor=SequentialEx()) + t_seq = @benchmark run!(mapping1, meteo_day; executor=SequentialEx()) #t_seq = run!(models1, meteo_day; executor = SequentialEx()) med_time_seq = median(t_seq).time #time is in nanoseconds @test med_time_seq > nrows * 1000000 - t_mt = @benchmark run!(models2, meteo_day; executor=ThreadedEx()) + t_mt = @benchmark run!(mapping2, meteo_day; executor=ThreadedEx()) #t_mt = run!(models2, meteo_day; executor = ThreadedEx()) med_time_mt = median(t_mt).time @@ -55,7 +55,7 @@ models2 = ModelList(process1=ToySleepModel(), status=(a=vc,)) #end # unsure how to recover outputs in benchmarked expressions to compare them, rerun the functions as a workaround for now - @test run!(models1, meteo_day; executor=SequentialEx()) == run!(models2, meteo_day; executor=ThreadedEx()) + @test run!(mapping1, meteo_day; executor=SequentialEx()) == run!(mapping2, meteo_day; executor=ThreadedEx()) end # TODO make sure a mt test with nthreads == 1 also is tested and is correct @@ -66,29 +66,27 @@ end using Dates meteo_day = read_weather(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), duration=Day) - models = ModelList( + mapping = ModelMapping( ToyLAIModel(), Beer(0.5), - status=(TT_cu=cumsum(meteo_day.TT),), + status=(TT_cu=cumsum(meteo_day.TT),) ) tracked_outputs = (:LAI,) - out_seq, out_mt = run_single_and_multi_thread_modellist(models, tracked_outputs, meteo_day) + out_seq, out_mt = run_single_and_multi_thread_modellist(mapping, tracked_outputs, meteo_day) @test compare_outputs_modellists(out_seq, out_mt) - modellists, status_tuples, outs_vectors = get_modellist_bank() + mappings_single_scale, _, outs_vectors = get_modelmapping_bank() meteos_all = get_simple_meteo_bank() # First meteo only has one timestep meteos = meteos_all[2:length(meteos_all)] - for i in 1:length(modellists) + for i in 1:length(mappings_single_scale) #i = 1 - modellist = modellists[i] - status_tuple = status_tuples[i] + mapping_template = mappings_single_scale[i] outs_vector = outs_vectors[i] - all_vars = init_variables(modellist) for j in 1:length(meteos) meteo = meteos[j] for k in 1:length(outs_vector) @@ -96,7 +94,8 @@ end out_tuple = outs_vector[k] try - out_st, out_mt = run_single_and_multi_thread_modellist(modellist, out_tuple, meteo) + mapping = deepcopy(mapping_template) + out_st, out_mt = run_single_and_multi_thread_modellist(mapping, out_tuple, meteo) @test compare_outputs_modellists(out_st, out_mt) catch e #print(i," ", j, " ", k) diff --git a/test/test-simulation.jl b/test/test-simulation.jl index 6813aa84d..96ff67eca 100644 --- a/test/test-simulation.jl +++ b/test/test-simulation.jl @@ -1,6 +1,6 @@ @testset "Check missing model" begin # No problem here: - @test_nowarn ModelList( + @test_nowarn ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -10,25 +10,58 @@ # Missing model for process2: @test_logs ( :info, - "Model Process3Model from process process3 needs a model that is a subtype of Process2Model in process process2, but the process is not parameterized in the ModelList." + "Model Process3Model from process process3 needs a model that is a subtype of Process2Model in process process2, but the process is not parameterized in the ModelMapping." ), ( :info, "Some variables must be initialized before simulation: (process3 = (:var5,),) (see `to_initialize()`)" ) - ModelList( + ModelMapping( process1=Process1Model(1.0), process3=Process3Model(), status=(var1=15.0, var2=0.3) ) end; +@testset "Deprecated run! entrypoints" begin + models = ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + status=(var1=15.0, var2=0.3) + ) + meteo = Atmosphere(T=20.0, Wind=1.0, Rh=0.65) + + run!(models, meteo) + @test_deprecated run!([models], meteo) + @test_throws ErrorException run!(ModelMapping("mod1" => models), meteo) + + mtg = Node(MultiScaleTreeGraph.NodeMTG("/", "Leaf", 1, 1)) + mtg[:var1] = 15.0 + mtg[:var2] = 0.3 + mapping_dict = Dict("Leaf" => (Process1Model(1.0), Process2Model(), Process3Model())) + @test_deprecated run!(mtg, mapping_dict, meteo) +end + +@testset "Single-scale multirate unsupported" begin + mapping = ModelMapping( + process1=Process1Model(1.0), + process2=Process2Model(), + process3=Process3Model(), + status=(var1=15.0, var2=0.3) + ) + meteo = Atmosphere(T=20.0, Wind=1.0, Rh=0.65) + + @test_throws "one-scale MTG" run!(mapping, meteo; multirate=true) + @test_throws "one-scale MTG" run!([mapping], meteo; multirate=true) +end + @testset "Simulation: 1 time-step, 0 Atmosphere" begin - models = ModelList( - Process1Model(1.0), + mapping = ModelMapping( + Process1Model(1.0); status=(var1=15.0, var2=0.3) ) - outputs = run!(models) + outputs = run!(mapping) vars = keys(outputs) @test [outputs[i][1] for i in vars] == [15.0, 0.3, 5.5] @@ -38,7 +71,7 @@ end; @testset "Simulation: 1 time-step, 1 Atmosphere" begin status_nt = (var1=15.0, var2=0.3) - models = ModelList( + mapping = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -47,23 +80,23 @@ end; meteo = Atmosphere(T=20.0, Wind=1.0, Rh=0.65) - modellist_outputs = run!(models, meteo) + modellist_outputs = run!(mapping, meteo) vars = keys(modellist_outputs) @test [modellist_outputs[i][1] for i in vars] == [34.95, 22.0, 56.95, 15.0, 5.5, 0.3] - mtg, mapping, out = check_multiscale_simulation_is_equivalent_begin(models, status_nt, meteo) - @test check_multiscale_simulation_is_equivalent_end(modellist_outputs, mtg, mapping, out, meteo) + mtg, mapping_mt, out = check_multiscale_simulation_is_equivalent_begin(mapping, meteo) + @test check_multiscale_simulation_is_equivalent_end(modellist_outputs, mtg, mapping_mt, out, meteo) end; @testset "Simulation: 1 time-step, 1 Atmosphere, 2 objects" begin - models = ModelList( + mapping = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), status=(var1=15.0, var2=0.3) ) - models2 = ModelList( + mapping2 = ModelMapping( process1=Process1Model(2.0), process2=Process2Model(), process3=Process3Model(), @@ -73,20 +106,20 @@ end; meteo = Atmosphere(T=20.0, Wind=1.0, Rh=0.65) @testset "simulation with an array of objects" begin - outputs_vector = run!([models, models2], meteo) + outputs_vector = run!([mapping, mapping2], meteo) @test [outputs_vector[1][i][1] for i in keys(outputs_vector[1])] == [34.95, 22.0, 56.95, 15.0, 5.5, 0.3] @test [outputs_vector[2][i][1] for i in keys(outputs_vector[2])] == [36.95, 26.0, 62.95, 15.0, 6.5, 0.3] end @testset "simulation with a dict of objects" begin - outputs_vector = run!(Dict("mod1" => models, "mod2" => models2), meteo) + outputs_vector = run!(Dict("mod1" => mapping, "mod2" => mapping2), meteo) @test [outputs_vector["mod1"][1][i] for i in keys(outputs_vector["mod1"])] == [34.95, 22.0, 56.95, 15.0, 5.5, 0.3] @test [outputs_vector["mod2"][1][i] for i in keys(outputs_vector["mod2"])] == [36.95, 26.0, 62.95, 15.0, 6.5, 0.3] end end; @testset "Simulation: 2 time-steps, 1 Atmosphere" begin - models = ModelList( + mapping = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -95,7 +128,7 @@ end; meteo = Atmosphere(T=20.0, Wind=1.0, Rh=0.65) - outputs = run!(models, meteo) + outputs = run!(mapping, meteo) vars = keys(outputs) @test [outputs[i] for i in vars] == [ [34.95, 35.550000000000004], @@ -111,7 +144,7 @@ end; status_nt = (var1=[15.0, 16.0], var2=0.3) - models = ModelList( + mapping = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), @@ -125,7 +158,7 @@ end; ] ) - modellist_outputs = run!(models, meteo) + modellist_outputs = run!(mapping, meteo) vars = keys(modellist_outputs) @test [modellist_outputs[i] for i in vars] == [ [34.95, 40.0], @@ -136,20 +169,20 @@ end; [0.3, 0.3], ] - mtg, mapping, out = check_multiscale_simulation_is_equivalent_begin(models, status_nt, meteo) - @test check_multiscale_simulation_is_equivalent_end(modellist_outputs, mtg, mapping, out, meteo) + mtg, mapping_mt, out = check_multiscale_simulation_is_equivalent_begin(mapping, meteo) + @test check_multiscale_simulation_is_equivalent_end(modellist_outputs, mtg, mapping_mt, out, meteo) end; @testset "Simulation: 2 time-steps, 2 Atmospheres, 2 objects" begin - models = ModelList( + mapping = ModelMapping( process1=Process1Model(1.0), process2=Process2Model(), process3=Process3Model(), status=(var1=[15.0, 16.0], var2=0.3) ) - models2 = ModelList( + mapping2 = ModelMapping( process1=Process1Model(2.0), process2=Process2Model(), process3=Process3Model(), @@ -164,7 +197,7 @@ end; ) @testset "simulation with an array of objects" begin - outputs_vector = run!([models, models2], meteo) + outputs_vector = run!([mapping, mapping2], meteo) @test [outputs_vector[1][i] for i in keys(outputs_vector[1])] == [ [34.95, 40.0], [22.0, 23.2], [56.95, 63.2], [15.0, 16.0], [5.5, 5.8], [0.3, 0.3] ] @@ -174,7 +207,7 @@ end; end @testset "simulation with a dict of objects" begin - outputs_vector = run!(Dict("mod1" => models, "mod2" => models2), meteo) + outputs_vector = run!(Dict("mod1" => mapping, "mod2" => mapping2), meteo) @test [[outputs_vector["mod1"][1][i], outputs_vector["mod1"][2][i]] for i in keys(outputs_vector["mod1"])] == [ [34.95, 40.0], [22.0, 23.2], [56.95, 63.2], [15.0, 16.0], [5.5, 5.8], [0.3, 0.3] ] @@ -191,7 +224,7 @@ end; leaf[:var1] = [15.0, 16.0] leaf[:var2] = 0.3 - mapping = Dict( + mapping = ModelMapping( "Leaf" => ( Process1Model(1.0), Process2Model(), @@ -223,15 +256,14 @@ end; end; -@testset "Meteo+ModelList/mapping+outputs combos either valid or different status vector size vs meteo length either run successfully or return a DimensionMisMatch" begin +@testset "Meteo+ModelMapping/mapping+outputs combos either valid or different status vector size vs meteo length either run successfully or return a DimensionMisMatch" begin verbose = false # set to true to print the indices of the combinations that fail meteos = get_simple_meteo_bank() - modellists, status_tuples, outputs_tuples_vectors = get_modellist_bank() + mappings_single_scale, _, outputs_tuples_vectors = get_modelmapping_bank() - for i in 1:length(modellists) + for i in 1:length(mappings_single_scale) # i = 3 - modellist = modellists[i] - status_tuple = status_tuples[i] + mapping_template = mappings_single_scale[i] outs_vector = outputs_tuples_vectors[i] for j in 1:length(meteos) @@ -241,7 +273,8 @@ end; # k = 7 out_tuple = outs_vector[k] @test try - outs_modellist = run!(modellist, meteo; tracked_outputs=out_tuple) + mapping = deepcopy(mapping_template) + outs_modellist = run!(mapping, meteo; tracked_outputs=out_tuple) true catch e verbose && print(i, " ", j, " ", k) @@ -293,4 +326,4 @@ end; end end end -end \ No newline at end of file +end diff --git a/test/test-toy_models.jl b/test/test-toy_models.jl index 7a4fd1354..f5d9f7386 100644 --- a/test/test-toy_models.jl +++ b/test/test-toy_models.jl @@ -3,33 +3,33 @@ meteo_day = CSV.read(joinpath(pkgdir(PlantSimEngine), "examples/meteo_day.csv"), # Note (smack) : The first test's behaviour is weird to me, because there is an [Info :] that correctly indicates # :LAI is not initialised, yet @test_nowarn doesn't capture it. I'm not sure what the intended test was, between 'Info' and 'Warn' @testset "ToyLAIModel" begin - @test_nowarn ModelList(ToyLAIModel()) - @test_nowarn ModelList(ToyLAIModel(), status=(TT_cu=10,)) - @test_nowarn ModelList( - ToyLAIModel(), + @test_nowarn ModelMapping(ToyLAIModel()) + @test_nowarn ModelMapping(ToyLAIModel(); status=(TT_cu=10,)) + @test_nowarn ModelMapping( + ToyLAIModel(); status=(TT_cu=cumsum(meteo_day.TT),), ) - m = ModelList( - ToyLAIModel(), + mapping = ModelMapping( + ToyLAIModel(); status=(TT_cu=cumsum(meteo_day.TT),), ) - outputs = @test_nowarn run!(m) + outputs = @test_nowarn run!(mapping) - @test m[:TT_cu] == cumsum(meteo_day.TT) + @test outputs[:TT_cu] == cumsum(meteo_day.TT) @test outputs[:LAI][begin] ≈ 0.00554987593080316 @test outputs[:LAI][end] ≈ 0.0 end @testset "ToyLAIModel+Beer" begin - models = ModelList( + mapping = ModelMapping( ToyLAIModel(), Beer(0.5), - status=(TT_cu=cumsum(meteo_day.TT),), + status=(TT_cu=cumsum(meteo_day.TT),) ) - outputs = run!(models, meteo_day) + outputs = run!(mapping, meteo_day) @test mean(outputs[:aPPFD]) ≈ 9.511021781482347 @test mean(outputs[:LAI]) ≈ 1.098492557536525 @@ -38,60 +38,56 @@ end @testset "ToyRUEGrowthModel" begin rue = 0.3 - @test_nowarn ModelList(ToyRUEGrowthModel(rue)) - @test_nowarn ModelList(ToyRUEGrowthModel(rue), status=(aPPFD=[10.0, 30.0, 25.0],)) + @test_nowarn ModelMapping(ToyRUEGrowthModel(rue)) + @test_nowarn ModelMapping(ToyRUEGrowthModel(rue); status=(aPPFD=[10.0, 30.0, 25.0],)) # One time step: - model = ModelList( - ToyRUEGrowthModel(rue), - status=(aPPFD=30.0,), - ) + mapping = ModelMapping(ToyRUEGrowthModel(rue); status=(aPPFD=30.0,)) - outputs = run!(model, executor=SequentialEx()) - @test outputs[:biomass][1] ≈ rue * model.status[:aPPFD] + outputs = run!(mapping, executor=SequentialEx()) + @test outputs[:biomass][1] ≈ rue * 30.0 # Several time steps: - model = ModelList( - ToyRUEGrowthModel(rue), - status=(aPPFD=[10.0, 30.0, 25.0],), - ) + aPPFD = [10.0, 30.0, 25.0] + mapping = ModelMapping(ToyRUEGrowthModel(rue); status=(aPPFD=aPPFD,)) - outputs = run!(model, executor=SequentialEx()) - @test outputs[:biomass] ≈ cumsum(rue * model.status[:aPPFD]) + outputs = run!(mapping, executor=SequentialEx()) + @test outputs[:biomass] ≈ cumsum(rue * aPPFD) end @testset "ToyAssimGrowthModel" begin - @test_nowarn ModelList(ToyAssimGrowthModel()) - @test_nowarn ModelList(ToyAssimGrowthModel(), status=(carbon_assimilation=[10.0, 30.0, 25.0],)) + @test_nowarn ModelMapping(ToyAssimGrowthModel()) + @test_nowarn ModelMapping(ToyAssimGrowthModel(); status=(carbon_assimilation=[10.0, 30.0, 25.0],)) # Uninitialized: - @test to_initialize(ModelList(ToyAssimGrowthModel())) == (growth=(:aPPFD,),) + to_init_uninitialized = to_initialize(ModelMapping(ToyAssimGrowthModel())) + if to_init_uninitialized isa AbstractDict + @test haskey(to_init_uninitialized, "Default") + @test :aPPFD in to_init_uninitialized["Default"] + else + @test :growth in keys(to_init_uninitialized) + @test :aPPFD in to_init_uninitialized[:growth] + end # One time step: - model = ModelList( - ToyAssimGrowthModel(), - status=(aPPFD=30.0,), - ) + mapping = ModelMapping(ToyAssimGrowthModel(); status=(aPPFD=30.0,)) - @test to_initialize(model) == NamedTuple() + @test isempty(to_initialize(mapping)) - outputs = run!(model) + outputs = run!(mapping) @test outputs[:biomass] ≈ [4.5] # Several time steps: - model = ModelList( - ToyAssimGrowthModel(), - status=(aPPFD=[10.0, 30.0, 25.0],), - ) + mapping = ModelMapping(ToyAssimGrowthModel(); status=(aPPFD=[10.0, 30.0, 25.0],)) - outputs = run!(model) + outputs = run!(mapping) @test outputs[:biomass] ≈ cumsum(outputs[:biomass_increment]) @test outputs[:biomass_increment] ≈ [0.8333333333333334, 4.5, 3.5833333333333335] end @testset "ToyLAIModel+Beer+ToyRUEGrowthModel" begin rue = 0.3 - models = ModelList( + mapping = ModelMapping( ToyLAIModel(), Beer(0.5), ToyRUEGrowthModel(rue), @@ -99,12 +95,12 @@ end ) # Match the warning on the executor, the default is ThreadedEx() but ToyRUEGrowthModel can't be run in parallel: - @test_logs (:warn, r"A parallel executor was provided") run!(models, meteo_day) + @test_logs (:warn, r"A parallel executor was provided") run!(mapping, meteo_day) # If we provide a serial executor, it works without a warning: - outputs = @test_nowarn run!(models, meteo_day, executor=SequentialEx()) + outputs = @test_nowarn run!(mapping, meteo_day, executor=SequentialEx()) @test mean(outputs[:aPPFD]) ≈ 9.511021781482347 @test mean(outputs[:LAI]) ≈ 1.098492557536525 @test outputs[:biomass][end] ≈ 1041.4687939085675 rtol = 1e-4 -end \ No newline at end of file +end