2158 add metplus ensemble stats capability by JorgeBornemann · Pull Request #2159 · MetOffice/CSET

JorgeBornemann (JorgeBornemann) · 2026-05-27T02:12:21Z

Introduces METPlus' EnsembleStat capability, with infrastructure based on PointStat. Fixes #2158

Contribution checklist

No AI has been used in this PR apart from web searches.

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

Documentation has been updated to reflect change.
New code has tests, and affected old tests have been updated.
All tests and CI checks pass.
Ensured the pull request title is descriptive.
Ensure rose-suite.conf.example has been updated if new diagnostic added.
Conda lock files have been updated if dependencies have changed.
Attributed any Generative AI, such as GitHub Copilot, used in this PR.
Marked the PR as ready to review.

github-actions · 2026-05-27T02:14:03Z

Total coverage: 91% (HTML report)

Name                                                              Stmts   Miss Branch BrPart  Cover
---------------------------------------------------------------------------------------------------
src/CSET/__init__.py                                                 93      2     12      0    98%
src/CSET/_common.py                                                 149      0     52      0   100%
src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py             116     28     26      0    77%
src/CSET/cset_workflow/app/finish_website/bin/finish_website.py      70      0      4      0   100%
src/CSET/cset_workflow/app/parbake_recipes/bin/parbake.py            29      0      8      0   100%
src/CSET/cset_workflow/app/send_email/bin/send_email.py              25      0      4      0   100%
src/CSET/cset_workflow/lib/python/jinja_utils.py                     17      0      6      0   100%
src/CSET/extract_workflow.py                                         47      0     16      0   100%
src/CSET/graph.py                                                    43      0     14      0   100%
src/CSET/operators/__init__.py                                       89      0     26      0   100%
src/CSET/operators/_atmospheric_constants.py                          9      0      0      0   100%
src/CSET/operators/_colormaps.py                                    229      4     62      4    97%
src/CSET/operators/_stash_to_lfric.py                                 3      0      0      0   100%
src/CSET/operators/_utils.py                                        183      8     74      6    95%
src/CSET/operators/ageofair.py                                      141      7     64      5    94%
src/CSET/operators/aggregate.py                                      76      1     22      1    98%
src/CSET/operators/aviation.py                                       60      0     18      0   100%
src/CSET/operators/collapse.py                                      154     12     72      5    91%
src/CSET/operators/constraints.py                                   111      7     48      2    93%
src/CSET/operators/convection.py                                     37      4     10      2    87%
src/CSET/operators/ensembles.py                                      27      0     14      0   100%
src/CSET/operators/feature.py                                        41      0     10      0   100%
src/CSET/operators/filters.py                                        66      2     30      0    98%
src/CSET/operators/fluxes.py                                         41      0     10      0   100%
src/CSET/operators/humidity.py                                      139      0     56      0   100%
src/CSET/operators/imageprocessing.py                                56      0     16      0   100%
src/CSET/operators/mesoscale.py                                      17      0      2      0   100%
src/CSET/operators/misc.py                                          158      1     66      4    98%
src/CSET/operators/plot.py                                         1011    180    346     67    78%
src/CSET/operators/power_spectrum.py                                 97      3     30      3    95%
src/CSET/operators/precipitation.py                                 176     34     84      4    80%
src/CSET/operators/pressure.py                                       41      0     12      0   100%
src/CSET/operators/read.py                                          409     38    178     14    89%
src/CSET/operators/regrid.py                                        122     18     64      1    83%
src/CSET/operators/scoreswrappers.py                                 47      6     12      3    85%
src/CSET/operators/temperature.py                                   121      0     32      0   100%
src/CSET/operators/transect.py                                       62      0     24      0   100%
src/CSET/operators/wind.py                                           45      3     10      2    91%
src/CSET/operators/write.py                                          15      0      6      0   100%
src/CSET/recipes/__init__.py                                        101      0     28      0   100%
---------------------------------------------------------------------------------------------------
TOTAL                                                              4473    358   1568    123    91%

Remove commented out lines for table name and database name.

Removed unused variables TABLE_NAME and DB_NAME from METPLUS configuration.

Removed TABLE_NAME and DB_NAME assignments for point_stat.

JorgeBornemann (JorgeBornemann) · 2026-07-07T04:24:05Z

This PR covers the implementation of METPlus ensemble stats apps and expanding the VerPy apps to cover ensemble stat cases. It sets up the scaffolding, based on 2 variables and producing RMSE and spread in the same plot, upon which the rest of variables and metrics will be added.

Scott Wales (ScottWales)

Thanks JorgeBornemann (@JorgeBornemann) I've added comments below.

Thinking bigger picture now that there's the skeleton of a few different met tools in CSET, what is the general user story for the verification? Are we expecting Metplus and Verpy to be pre-set with some basic stats that users generally don't modify, or will users be selecting the fields and statistics they're plotting in the CSET configuration?

Scott Wales (ScottWales) · 2026-07-07T04:24:46Z

+[runtime]
+    [[metplus_prep_obs]]
+        inherit = METPLUS
+
+{% for model in models %}
+    [[metplus_prep_fcst_m{{model["id"]}}]]
+    # Prepares model data for use by metplus
+        inherit = METPLUS
+        [[[environment]]]
+            ROSE_TASK_APP = metplus_prep_fcst
+            MODEL_NAME = {{model["name"]}}
+            MODEL_ID = m{{model["id"]}}
+            INPUT_PATHS = $CYLC_TASK_SHARE_CYCLE_DIR/data/{{model["id"]}}
+        {% if SELECT_SUBAREA %}
+            SUBAREA_TYPE = {{SUBAREA_TYPE}}
+            SUBAREA_EXTENT = {{SUBAREA_EXTENT}}
+        {% else %}
+            SUBAREA_TYPE = None
+            SUBAREA_EXTENT = None
+        {% endif %}
+{% endfor %}
+
+    [[metplus_ascii2nc]]
+    # Runs METplus wrappers for ASCII to nc ingestion of obs.
+        inherit = METPLUS
+        [[[environment]]]
+            ROSE_APP_OPT_CONF_KEYS = {{SITE}}_ens


Is this duplicating sections from point_stat that would be better off in metplus_common.cylc?

I have moved the metplus_prep_fcst_m{{model["id"]}} definition to the common include in commit 3acae3e, leaving metplus_prep_obs and metplus_ascii2nc in place as they are not shared with grid-stat.

For metplus_ascii2nc, what happens if both RUN_METPLUS_POINT_STAT and RUN_METPLUS_ENSMBLE_STAT are true?

It will be included twice, the second include overwriting the first one, which will be a problem. I don't envisage a run of the cset workflow used for both ensemble and deterministic, shall we make it mutually exclusive in the metadata with a ' if either RUN_METPLUS_POINT_STAT or RUN_METPLUS_GRID_STAT is True RUN_METPLUS_ENSEMBLE_STAT cannot be True' rule?

That sounds like a good idea for the time being

or rename the ensemble one as metplus_ascii2nc_ens or something like that

After fighting with the metadata for a while I went for splitting metplus_ascii2nc for ensemble_stat, 15d7c77.
Note that the metadata for METPlus tasks requires a wholesale revamp.

…om:MetOffice/CSET into 2158-add-metplus-ensemble_stats-capability

JorgeBornemann (JorgeBornemann) · 2026-07-13T03:47:50Z

Thanks Scott Wales (@ScottWales) . I have responded to your comments. Regarding the general comment about whether there will be flexibility in the METPlus apps or they will be based in fixed configurations, the intention is to have some flexibility, being able to provide requests, such as variables or statistics, via the GUI, plus a number of pre-set configurations (i.e. for RAL4 package evaluations). At the moment the priority is to provide a CSET with METPlus configuration working for RAL4 evaluation, hence not looking much into the flexibility aspect, but there are already some ideas omn how to implement it.

Scott Wales (ScottWales) · 2026-07-15T23:27:35Z

Thanks JorgeBornemann (@JorgeBornemann), just a last concern on if the different metplus_ascii2nc definitions for point_stat and ensemble_stat cause conflicts

JorgeBornemann (JorgeBornemann) added 2 commits May 27, 2026 01:56

Add graphing and rose-meta options for ensemble stats

1b48157

add ensemble stat app

88f4298

JorgeBornemann (JorgeBornemann) self-assigned this May 27, 2026

JorgeBornemann (JorgeBornemann) added enhancement New feature or request metplus labels May 27, 2026

JorgeBornemann (JorgeBornemann) linked an issue May 27, 2026 that may be closed by this pull request

Add METPlus Ensemble_Stats capability #2158

Open

JorgeBornemann (JorgeBornemann) marked this pull request as draft May 27, 2026 02:13

JorgeBornemann (JorgeBornemann) added 2 commits July 7, 2026 03:09

add verpy config changes for ensemble stats

6d9979e

Resolve conflicts with MAIN

38816f0

JorgeBornemann (JorgeBornemann) marked this pull request as ready for review July 7, 2026 03:45

JorgeBornemann (JorgeBornemann) added 4 commits July 7, 2026 15:48

remove unneeded comments

817d1cb

Remove unnecessary comments

3eff94d

Remove commented out lines for table name and database name.

Remove unused vars

dd070ff

Removed unused variables TABLE_NAME and DB_NAME from METPLUS configuration.

Remove uused Vars

518804f

Removed TABLE_NAME and DB_NAME assignments for point_stat.

JorgeBornemann (JorgeBornemann) requested review from Scott Wales (ScottWales) and James Frost (jfrost-mo) July 7, 2026 04:02

Scott Wales (ScottWales) requested changes Jul 7, 2026

View reviewed changes

JorgeBornemann (JorgeBornemann) added 3 commits July 7, 2026 04:46

add changes to ascii2nc for niwa ensemble models

817085f

Merge branch '2158-add-metplus-ensemble_stats-capability' of github.c…

475c1df

…om:MetOffice/CSET into 2158-add-metplus-ensemble_stats-capability

Implement code review requests

3acae3e

JorgeBornemann (JorgeBornemann) added 2 commits July 16, 2026 04:38

correct metadata trigger from ensemble stat

d15315a

split ascii2nc task def for ensemble stats

15d7c77

Scott Wales (ScottWales) approved these changes Jul 16, 2026

View reviewed changes

Uh oh!

Conversation

JorgeBornemann (JorgeBornemann) commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution checklist

Uh oh!

github-actions Bot commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JorgeBornemann (JorgeBornemann) commented Jul 7, 2026

Uh oh!

Scott Wales (ScottWales) left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Scott Wales (ScottWales) Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

JorgeBornemann (JorgeBornemann) Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Scott Wales (ScottWales) Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

JorgeBornemann (JorgeBornemann) Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

Scott Wales (ScottWales) Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

Scott Wales (ScottWales) Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

JorgeBornemann (JorgeBornemann) Jul 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JorgeBornemann (JorgeBornemann) commented Jul 13, 2026

Uh oh!

Scott Wales (ScottWales) commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

JorgeBornemann (JorgeBornemann) commented May 27, 2026 •

edited

Loading

github-actions Bot commented May 27, 2026 •

edited

Loading