Skip to content

Add RMSE scores based vertical pressure profile recipe#2230

Open
Sylvia Bohnenstengel (Sylviabohnenstengel) wants to merge 15 commits into
mainfrom
2229_scores_rmse_vertical_profile_pressure_levels
Open

Add RMSE scores based vertical pressure profile recipe#2230
Sylvia Bohnenstengel (Sylviabohnenstengel) wants to merge 15 commits into
mainfrom
2229_scores_rmse_vertical_profile_pressure_levels

Conversation

@Sylviabohnenstengel

@Sylviabohnenstengel Sylvia Bohnenstengel (Sylviabohnenstengel) commented Jun 24, 2026

Copy link
Copy Markdown
Member

addresses part of #2229
1st PR in a series of PRS.
Inludes RMSE profiles per case whole case study and per timepoint on pressure levels.

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

  • Documentation has been updated to reflect change.
  • New code has tests, and affected old tests have been updated.
  • All tests and CI checks pass.
  • Ensured the pull request title is descriptive.
  • Ensure rose-suite.conf.example has been updated if new diagnostic added.
  • Attributed any Generative AI, such as GitHub Copilot, used in this PR.
  • Marked the PR as ready to review.

…otting of vertical profiles without sequence_coordinate for 1D cubes.
@github-actions

github-actions Bot commented Jun 24, 2026

Copy link
Copy Markdown
Contributor
Total coverage: 91% (HTML report)
Name                                                              Stmts   Miss Branch BrPart  Cover
---------------------------------------------------------------------------------------------------
src/CSET/__init__.py                                                 93      2     12      0    98%
src/CSET/_common.py                                                 149      0     52      0   100%
src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py             116     28     26      0    77%
src/CSET/cset_workflow/app/finish_website/bin/finish_website.py      70      0      4      0   100%
src/CSET/cset_workflow/app/parbake_recipes/bin/parbake.py            29      0      8      0   100%
src/CSET/cset_workflow/app/send_email/bin/send_email.py              25      0      4      0   100%
src/CSET/cset_workflow/lib/python/jinja_utils.py                     17      0      6      0   100%
src/CSET/extract_workflow.py                                         47      0     16      0   100%
src/CSET/graph.py                                                    43      0     14      0   100%
src/CSET/operators/__init__.py                                       89      0     26      0   100%
src/CSET/operators/_atmospheric_constants.py                          9      0      0      0   100%
src/CSET/operators/_colormaps.py                                    230      1     62      2    99%
src/CSET/operators/_stash_to_lfric.py                                 3      0      0      0   100%
src/CSET/operators/_utils.py                                        193      8     76      6    95%
src/CSET/operators/ageofair.py                                      141      7     64      5    94%
src/CSET/operators/aggregate.py                                      76      1     22      1    98%
src/CSET/operators/aviation.py                                       60      0     18      0   100%
src/CSET/operators/collapse.py                                      154      8     72      3    93%
src/CSET/operators/constraints.py                                   111      7     48      2    93%
src/CSET/operators/convection.py                                     37      4     10      2    87%
src/CSET/operators/ensembles.py                                      27      0     14      0   100%
src/CSET/operators/feature.py                                        41      0     10      0   100%
src/CSET/operators/filters.py                                        66      2     30      0    98%
src/CSET/operators/fluxes.py                                         41      0     10      0   100%
src/CSET/operators/humidity.py                                      139      0     56      0   100%
src/CSET/operators/imageprocessing.py                                56      0     16      0   100%
src/CSET/operators/mesoscale.py                                      17      0      2      0   100%
src/CSET/operators/misc.py                                          158      1     66      4    98%
src/CSET/operators/plot.py                                         1013    166    346     63    80%
src/CSET/operators/power_spectrum.py                                 97      3     30      3    95%
src/CSET/operators/precipitation.py                                 176     34     84      4    80%
src/CSET/operators/pressure.py                                       41      0     12      0   100%
src/CSET/operators/read.py                                          422     41    184     14    89%
src/CSET/operators/regrid.py                                        122     18     64      1    83%
src/CSET/operators/scoreswrappers.py                                106     16     36      5    81%
src/CSET/operators/temperature.py                                   121      0     32      0   100%
src/CSET/operators/transect.py                                       62      0     24      0   100%
src/CSET/operators/wind.py                                           45      3     10      2    91%
src/CSET/operators/write.py                                          15      0      6      0   100%
src/CSET/recipes/__init__.py                                        101      0     28      0   100%
---------------------------------------------------------------------------------------------------
TOTAL                                                              4558    350   1600    117    91%

@Sylviabohnenstengel

Copy link
Copy Markdown
Member Author

Test figure for RMSE testing MODELB vs MODELB expecting RMSE=0.0 on all levels.
image

…tial field and preserve time as sequence coordinate to slide over. This required an addition to scoreswrappers.py that maps the xarray time dimensions with iris auxilliary time dimension requirements. The scoreswrapper was developed with help of Github copilot.
@Sylviabohnenstengel

Sylvia Bohnenstengel (Sylviabohnenstengel) commented Jun 24, 2026

Copy link
Copy Markdown
Member Author

Added capability to calculate the RMSE over the spatial domain on each layer for each timestep separately. This required a change to scores wrapper that matches the expected preserved dimensions that match xarray dimension names with the iris coordinates such as "time". The latter might be auxiliary coordinates attached to a differently named dimension such as dim0 so these needed matching.

Adding plots to show that recipe works for individual times as well.
This requires the addition of "time" to the preserved coordinate in the recipe when calling the scores_rmse wrapper.

image

@Sylviabohnenstengel Sylvia Bohnenstengel (Sylviabohnenstengel) added enhancement New feature or request full_review Requires a technical, scientific, and portability review labels Jun 25, 2026
@Sylviabohnenstengel Sylvia Bohnenstengel (Sylviabohnenstengel) linked an issue Jun 25, 2026 that may be closed by this pull request
10 tasks
…or rmse_scores_profile when aggregating over an entire case study's time coordinate resulting in a single rmse profile.
@Sylviabohnenstengel

Sylvia Bohnenstengel (Sylviabohnenstengel) commented Jun 25, 2026

Copy link
Copy Markdown
Member Author

Added aggregation period into title of profile RMSE plot. vertical plotting profile now recognises scalar sequence coordinates and uses them in the title/filename. In addition the output filename now includes the time range over which the RMSE was calculated.

Github copilot utilised for this commit.

Figures now shows the aggregation period when only preserving pressure and not time in the rmse calculation with scores.
image

If intending to plot individual rmse profiles for evevry time point then "time" needs to be added alongside "pressure" in the recipe when calling rmse scores. Figure below shows RMSe workign with date and time in title for a snapshot as well.
image

…teps in a case study and in additional rmse profile for evevry single time step.
… (sequence) and vertical profile RMSE over whole case. In addition identifying base_model at start of loader.
@Sylviabohnenstengel

Copy link
Copy Markdown
Member Author

vertical profiles of RMSE for a sequence such as time points as well as vertical profile of RMSE for whole case study are enabled independently of each other in the rose gui.

image

…rity for basis of RMSE calculation for profiles.
@Sylviabohnenstengel Sylvia Bohnenstengel (Sylviabohnenstengel) changed the title Add RMSE scores based vertical pressure profile recipe and include pl… Add RMSE scores based vertical pressure profile recipe Jul 16, 2026
@Sylviabohnenstengel

Copy link
Copy Markdown
Member Author
image image

@Sylviabohnenstengel

Copy link
Copy Markdown
Member Author

layout of rose-suite.conf and rose-suite.meta for verification. This likely will get merge conflicts with #2232

image

@Sylviabohnenstengel
Sylvia Bohnenstengel (Sylviabohnenstengel) marked this pull request as ready for review July 16, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request full_review Requires a technical, scientific, and portability review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adding SCORES RMSE functionality to profiles.

2 participants