Skip to content

Commit 9e3f990

Browse files
authored
Merge branch 'develop' into GH-3710-fix-broken-documentation-links
2 parents 866a257 + 2fce365 commit 9e3f990

66 files changed

Lines changed: 2396 additions & 311 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ For more information about this file see also [Keep a Changelog](http://keepacha
4141
`MCMCpack`, `mvtnorm`, `neonUtilities`, `neonstore`, `PEcAn.benchmark`,
4242
`PEcAn.visualization`, `rjags`, `sirt`, and `sp` from `Imports` to
4343
`Suggests` (@omkarrr2533, #3599).
44+
- Management events specified via `events.json` are now required to specify a crop code for each planting event, so that models can know when to restart with a different PFT (#3828, #3836).
4445

4546

4647

apps/api/R/posteriors.R

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ searchPosteriors <- function(req, pft_id = NA, host_id = NA,
104104
result$prev_page <- paste0(
105105
req$rook.url_scheme, "://",
106106
req$HTTP_HOST,
107-
"/api/workflows",
107+
"/api/posteriors",
108108
req$PATH_INFO,
109109
substr(req$QUERY_STRING,
110110
0,

apps/api/R/workflows.R

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -312,7 +312,9 @@ getWorkflowFilesAsZip <- function(req, id, filenames, res){
312312

313313
full_files[i] <- filepath
314314
}
315-
zip_file <- zip::zipr("output.zip", full_files)
316-
return(zip_file)
315+
zip_path <- zip::zipr("output.zip", full_files)
316+
bin <- readBin(zip_path, "raw", n = file.info(zip_path)$size)
317+
file.remove(zip_path)
318+
return(bin)
317319
}
318320
}

base/db/R/dbfiles.R

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -766,7 +766,7 @@ dbfile.move <- function(old.dir, new.dir, file.type, siteid = NULL, register = F
766766
files.indb <- 0
767767

768768
# check for file type and update to make it *.file type
769-
if (file.type != "clim" | file.type != "nc") {
769+
if (file.type != "clim" && file.type != "nc") {
770770
PEcAn.logger::logger.error("File type not supported by move at this time. Currently only supports NC and CLIM files")
771771
error <- 1
772772
}

base/db/R/get.trait.data.pft.R

Lines changed: 45 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,54 @@
11
##' Get trait data from the database for a single PFT
22
##'
3-
##' @details `pft` should be a list containing at least `name` and `outdir`, and optionally `posteriorid` and `constants`. BEWARE: All existing files in `outir` will be deleted!
4-
##' @param pft list of settings for the pft whose traits to retrieve. See details
5-
##' @param modeltype type of model that is used, this is used to distinguish between different pfts with the same name.
3+
##' @md
4+
##' Queries BETYdb for trait observations and prior distributions for a single
5+
##' plant functional type (PFT). Results are saved to files
6+
##' in the PFT output directory (`pft$outdir`), and also registered in the
7+
##' database as posterior records when `write = TRUE`.
8+
##'
9+
##' @details
10+
##' `pft` should be a list containing at least `name` and `outdir`, and
11+
##' optionally `posteriorid` and `constants`.
12+
##'
13+
##' **File-based side effects (saved to `pft$outdir`):**
14+
##' \describe{
15+
##' \item{`trait.data.Rdata`}{Contains a single object `trait.data`: a named
16+
##' list of data frames, one per trait. Each data frame has columns from
17+
##' BETYdb's traits/yields views (e.g., `mean`, `stat`, `n`, `site_id`,
18+
##' `treatment_id`). Names correspond to trait variable names
19+
##' (e.g., `"SLA"`, `"Vcmax"`).}
20+
##' \item{`prior.distns.Rdata`}{Contains a single object `prior.distns`: a
21+
##' data frame with one row per trait and columns `distn`, `parama`,
22+
##' `paramb`, and `n`. Row names are trait variable names. Traits listed
23+
##' in `pft$constants` are excluded.}
24+
##' \item{`trait.data.csv`}{CSV export of `trait.data` (all traits
25+
##' row-bound).}
26+
##' \item{`prior.distns.csv`}{CSV export of `prior.distns`.}
27+
##' \item{`species.csv` or `cultivars.csv`}{PFT membership list used to
28+
##' detect changes between runs.}
29+
##' }
30+
##'
31+
##' **Downstream contract:** The files `trait.data.Rdata` and
32+
##' `prior.distns.Rdata` are expected by \code{run.meta.analysis.pft}, which
33+
##' loads them from `pft$outdir`. This implicit file-based coupling means
34+
##' the two functions must agree on directory path and object names. A future
35+
##' refactoring goal is to pass these objects directly via function arguments
36+
##' instead.
37+
##'
38+
##' @param pft list of settings for the pft whose traits to retrieve. See details.
39+
##' @param modeltype type of model that is used, this is used to distinguish
40+
##' between different pfts with the same name.
641
##' @param dbfiles location where previous results are found
742
##' @param dbcon database connection
8-
##' @param forceupdate set this to true to force an update, auto will check to see if an update is needed.
43+
##' @param forceupdate set this to true to force an update, auto will check to
44+
##' see if an update is needed.
945
##' @param write (Logical) If `TRUE` updated posteriors will be written to
10-
##' BETYdb. Defaults to FALSE.
46+
##' BETYdb. Defaults to `FALSE`.
1147
##' @param trait.names list of trait names to retrieve
12-
##' @return updated pft with posteriorid
48+
##' @return The `pft` input list, updated with `pft$posteriorid` set to the
49+
##' ID of the (possibly new) posterior record in BETYdb. The posterior ID can
50+
##' be used to locate the output files (`trait.data.Rdata`, `prior.distns.Rdata`,
51+
##' etc.) via BETYdb's `dbfiles` table.
1352
##' @author David LeBauer, Shawn Serbin, Rob Kooper
1453
##' @export
1554
get.trait.data.pft <-
@@ -27,10 +66,6 @@ get.trait.data.pft <-
2766
PEcAn.logger::logger.error(paste0("Couldn't create PFT output directory: ", pft$outdir))
2867
}
2968

30-
## Remove old files. Clean up.
31-
old.files <- list.files(path = pft$outdir, full.names = TRUE, include.dirs = FALSE)
32-
file.remove(old.files)
33-
3469
# find appropriate pft
3570
pftres <- query_pfts(dbcon, pft[["name"]], modeltype)
3671
pfttype <- pftres[["pft_type"]]

base/db/man/get.trait.data.pft.Rd

Lines changed: 38 additions & 7 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

base/remote/R/check_model_run.R

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,9 @@ check_model_run <- function(out, stop.on.error = TRUE) {
1010
success <- FALSE
1111
msg <- paste0("Model run aborted with the following error:\n", out)
1212
if (stop.on.error) {
13-
PEcAn.logger::logger.severe(msg)
13+
PEcAn.logger::logger.severe(msg, wrap = FALSE)
1414
} else {
15-
PEcAn.logger::logger.error(msg)
15+
PEcAn.logger::logger.error(msg, wrap = FALSE)
1616
}
1717
} else {
1818
success <- TRUE

base/workflow/R/run.write.configs.R

Lines changed: 50 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,62 @@
11
#' Write model-specific run scripts and configuration files
22
#'
3-
#' Generates run scripts and configuration files for all analyses specified
4-
#' in the provided settings. Most of the heavy lifting is done by the
5-
#' \code{write.config.*} function for your specific ecosystem model
6-
#' (e.g. write.config.ED2, write.config.SIPNET).
3+
#' @md
4+
#' Generates run scripts and configuration files for all analyses (ensemble
5+
#' and/or sensitivity analysis) specified in the provided settings. Delegates
6+
#' the model-specific config writing to the appropriate `write.config.*`
7+
#' function (e.g. `write.config.ED2`, `write.config.SIPNET`).
78
#'
9+
#' @details
10+
#' **Upstream contract (reads from `settings$outdir`):**
11+
#' \describe{
12+
#' \item{`samples.Rdata`}{Produced by \code{\link[PEcAn.uncertainty]{get.parameter.samples}}.
13+
#' Contains 5 bundled objects: `trait.samples`,
14+
#' `sa.samples`, `ensemble.samples`, `runs.samples`, `env.samples`.
15+
#' This function loads `trait.samples` and `sa.samples` to build
16+
#' model configuration files. If `input_design` contains a `param`
17+
#' column, `ensemble.samples` is rebuilt by subsetting `trait.samples`
18+
#' according to the design indices.}
19+
#' }
20+
#'
21+
#' **File-based side effects (saved to `settings$outdir`):**
22+
#' \describe{
23+
#' \item{`sensitivity.samples.<ensemble_id>.Rdata`}{Contains `sa.run.ids`
24+
#' (named list of run IDs per PFT/trait/quantile), `sa.ensemble.id`,
25+
#' `sa.samples`, `pft.names`, and `trait.names`. Saved when sensitivity
26+
#' analysis is configured.}
27+
#' \item{`ensemble.samples.<ensemble_id>.Rdata`}{Contains `ens.run.ids`
28+
#' (vector of run IDs), `ens.ensemble.id`, `ens.samples`, `pft.names`,
29+
#' and `trait.names`. Saved when ensemble is configured.}
30+
#' \item{`runs_manifest.csv`}{A CSV table tracking all runs created,
31+
#' appended across ensemble and SA analyses.}
32+
#' }
33+
#'
34+
#' **Downstream contract:** The `sensitivity.samples.*.Rdata` and
35+
#' `ensemble.samples.*.Rdata` files are loaded by \code{\link[PEcAn.uncertainty]{get.results}}
36+
#' to match model outputs to their corresponding
37+
#' parameter sets. This implicit file-based coupling is a refactoring target.
38+
#'
39+
#' The default value for `posterior.files` is NA, in which case the
40+
#' most recent posterior or prior (in that order) for the workflow is used.
41+
#' When specified, `posterior.files` should be a vector of filenames with one
42+
#' entry for each PFT. Specify filenames with no path; PFT outdirs will be
43+
#' appended. This forces use of only files within this workflow, to avoid
44+
#' confusion.
845
#'
946
#' @param settings a PEcAn settings list
1047
#' @param ensemble.size number of ensemble runs
11-
#' @param input_design Input design data.frame coordinating input files across runs.
12-
#' Contains columns for each sampled input (met, param, etc.) with row indices,
13-
#' as documented in \code{runModule.run.write.configs()}.
48+
#' @param input_design Input design data.frame coordinating input files across
49+
#' runs. Contains columns for each sampled input (met, param, etc.) with row
50+
#' indices, as documented in \code{\link[PEcAn.workflow]{runModule.run.write.configs}}.
1451
#' @param write should the runs be written to the database?
15-
#' @param posterior.files Filenames for posteriors for drawing samples for ensemble and sensitivity
16-
#' analysis (e.g. post.distns.Rdata, or prior.distns.Rdata)
52+
#' @param posterior.files Filenames for posteriors for drawing samples for
53+
#' ensemble and sensitivity analysis (e.g. `post.distns.Rdata`, or
54+
#' `prior.distns.Rdata`).
1755
#' @param overwrite logical: Replace output files that already exist?
1856
#'
19-
#' @details The default value for \code{posterior.files} is NA, in which case the
20-
#' most recent posterior or prior (in that order) for the workflow is used.
21-
#' When specified, \code{posterior.files} should be a vector of filenames with one entry for each PFT.
22-
#' Specify filenames with no path; PFT outdirs will be appended. This forces use of only
23-
#' files within this workflow, to avoid confusion.
24-
#'
25-
#' @return an updated settings list, which includes ensemble IDs for SA and ensemble analysis
57+
#' @return The `settings` list (invisibly), updated with ensemble IDs for SA
58+
#' and ensemble analysis (e.g. `settings$sensitivity.analysis$ensemble.id`,
59+
#' `settings$ensemble$ensemble.id`).
2660
#' @export
2761
#'
2862
#' @author David LeBauer, Shawn Serbin, Ryan Kelly, Mike Dietze, Akash B V

base/workflow/man/run.write.configs.Rd

Lines changed: 44 additions & 14 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)