Add Scores diagnostics for spatial and timeseries RMSE, Additive Bias, MAE, and Pearson’s Correlation Coefficient by BernardClaxton · Pull Request #2232 · MetOffice/CSET

BernardClaxton (BernardClaxton) · 2026-06-25T12:05:16Z

Contribution checklist

Aim to have all relevant checks ticked off before merging. See the developer's guide for more detail.

Documentation has been updated to reflect change.
New code has tests, and affected old tests have been updated.
All tests and CI checks pass.
Ensured the pull request title is descriptive.
Ensure rose-suite.conf.example has been updated if new diagnostic added.
Attributed any Generative AI, such as GitHub Copilot, used in this PR.
Marked the PR as ready to review.

github-actions · 2026-06-25T12:07:14Z

Total coverage: 90% (HTML report)

Name                                                              Stmts   Miss Branch BrPart  Cover
---------------------------------------------------------------------------------------------------
src/CSET/__init__.py                                                 93      2     12      0    98%
src/CSET/_common.py                                                 149      0     52      0   100%
src/CSET/cset_workflow/app/fetch_fcst/bin/fetch_data.py             115     27     24      0    79%
src/CSET/cset_workflow/app/finish_website/bin/finish_website.py      70      0      4      0   100%
src/CSET/cset_workflow/app/parbake_recipes/bin/parbake.py            29      0      8      0   100%
src/CSET/cset_workflow/app/send_email/bin/send_email.py              25      0      4      0   100%
src/CSET/cset_workflow/lib/python/jinja_utils.py                     17      0      6      0   100%
src/CSET/extract_workflow.py                                         47      0     16      0   100%
src/CSET/graph.py                                                    43      0     14      0   100%
src/CSET/operators/__init__.py                                       89      0     26      0   100%
src/CSET/operators/_atmospheric_constants.py                          9      0      0      0   100%
src/CSET/operators/_colormaps.py                                    229      4     62      4    97%
src/CSET/operators/_stash_to_lfric.py                                 3      0      0      0   100%
src/CSET/operators/_utils.py                                        183      8     74      6    95%
src/CSET/operators/ageofair.py                                      141      7     64      5    94%
src/CSET/operators/aggregate.py                                      76      1     22      1    98%
src/CSET/operators/aviation.py                                       60      0     18      0   100%
src/CSET/operators/collapse.py                                      154     12     72      5    91%
src/CSET/operators/constraints.py                                   111      7     48      2    93%
src/CSET/operators/convection.py                                     37      4     10      2    87%
src/CSET/operators/ensembles.py                                      27      0     14      0   100%
src/CSET/operators/feature.py                                        41      0     10      0   100%
src/CSET/operators/filters.py                                        66      2     30      0    98%
src/CSET/operators/fluxes.py                                         41      0     10      0   100%
src/CSET/operators/humidity.py                                      139      0     56      0   100%
src/CSET/operators/imageprocessing.py                                56      0     16      0   100%
src/CSET/operators/mesoscale.py                                      17      0      2      0   100%
src/CSET/operators/misc.py                                          158      1     66      4    98%
src/CSET/operators/plot.py                                         1011    180    346     67    78%
src/CSET/operators/power_spectrum.py                                 97      3     30      3    95%
src/CSET/operators/precipitation.py                                 176     34     84      4    80%
src/CSET/operators/pressure.py                                       41      0     12      0   100%
src/CSET/operators/read.py                                          409     38    178     14    89%
src/CSET/operators/regrid.py                                        122     18     64      1    83%
src/CSET/operators/scoreswrappers.py                                135     22     34      8    79%
src/CSET/operators/temperature.py                                   121      0     32      0   100%
src/CSET/operators/transect.py                                       62      0     24      0   100%
src/CSET/operators/wind.py                                           45      3     10      2    91%
src/CSET/operators/write.py                                          15      0      6      0   100%
src/CSET/recipes/__init__.py                                        101      0     28      0   100%
---------------------------------------------------------------------------------------------------
TOTAL                                                              4560    373   1588    128    90%

BernardClaxton (BernardClaxton) · 2026-06-25T13:59:24Z

The test scripts test_scoreswrappers.py and test_verification.py were written with the aid of Copilot.

BernardClaxton (BernardClaxton) · 2026-06-25T14:37:39Z

Output from testing the scores utility can be found at https://wwwspice/~bernie.claxton/CSET_rmse8/?q=scores . The scores plots all have "Scores" in the title.

Sylvia Bohnenstengel (Sylviabohnenstengel) · 2026-07-14T17:50:24Z

BernardClaxton (@BernardClaxton) in scoreswrappers all scores will need to deal with the preserved_coordinates in the same way as in the rmse calculation as they are currently failing in my test case. This change is mostly a copy and paste.

…ores error metrics for spatial surface plots.

Sylvia Bohnenstengel (Sylviabohnenstengel) · 2026-07-15T09:26:36Z

BernardClaxton (@BernardClaxton) I have check out your branch and added the handling and conversion of iris dim coordinates to xarray dim coordinates as expected by scores to the surface scores plots.

Sylvia Bohnenstengel (Sylviabohnenstengel) · 2026-07-15T09:51:21Z

+        for model, field, method, scores_method in itertools.product(
+            models[1:],
+            conf.SURFACE_FIELDS,
+            conf.SPATIAL_SURFACE_FIELD_METHOD,


following discussion with BernardClaxton (@BernardClaxton) I suggest that we include the separate methods selection for verification in this PR and the inclusion of a list of variables that the verification uses will be a separate PR to avoid moving the goalposts.

Sylvia Bohnenstengel (Sylviabohnenstengel) · 2026-07-15T11:05:11Z

+  for all times. The PC is calculated based on that used in the
+  package [`scores`](https://scores.readthedocs.io/en/stable/api.html#scores.continuous.correlation.pearsonr).
+  This recipe allows the preservation of the time coordinate to produce a timeseries. Therefore, the PC is
+  collapsed over all other coordinates in the cube and calculated for every timestep.


they should not be collapsed, but handled directly in scores utilising the preserved_coordinates.

…AME #2214

BernardClaxton (BernardClaxton) · 2026-07-15T13:03:19Z

Added support in the scores loaders and recipes for SUBAREA_NAME.

BernardClaxton (BernardClaxton) · 2026-07-15T13:38:46Z

Added SPATIAL_SCORES_FIELD_METHOD into the spatial scores GUI. This replaces the previous reliance on SPATIAL_SURFACE_FIELD_METHOD found under the Diagnostics / Surface (2D) fields tab. Possible values for this list are SEQ, CASE, MAX, MIN. Selecting CASE will instruct CSET to compute the spatial scores metrics over the entire case study.

Co-authored-by: Sylvia Bohnenstengel <62748926+Sylviabohnenstengel@users.noreply.github.com>

…ve_bias.yaml Co-authored-by: Sylvia Bohnenstengel <62748926+Sylviabohnenstengel@users.noreply.github.com>

BernardClaxton (BernardClaxton) · 2026-07-15T14:18:02Z

Sample workflow output for branch 2214_adding_scores can be found at https://wwwspice/~bernie.claxton/CSET_scores20/?q=scores

BernardClaxton (BernardClaxton) added 18 commits June 24, 2026 15:53

Loaders for new Scores metrics #2214

09490d1

Recipe for 20d spatial plots of scores metrics #2214

63e12f5

Recipe for 2-d timeseries plots of scores metrics #2214

68ee904

Operator for scores metrics #2214

4ba0e65

Rose GUI edits for scorer metrics #2214

7229736

Tidy loaders for scorer metrics #2214

ab7c205

Tidy recipe for spatial plots of scorer metrics #2214

6100386

Tidy recipe for timeseries plots of scorer metrics #2214

81b21dc

Tidy operator for scorer metrics #2214

f136481

Tidy comments in operator for scorer metrics #2214

0d2a5d5

Changed the Rose GUI for the scores metrics #2214

67dcd2d

Changed the scores loaders to reflect the Rose GUI redesign #2214

a658473

Tidied the logic in scores loaders #2214

391574d

Added scores Pearson Correlation timeseries to Rose GUI #2214

a66c7ad

Added scores Pearson Correlation timeseries to loader #2214

91277e4

Removed bmc from titles of spatial plots #2214

f9ebf3f

Removed bmc from titles of timeseries plots #2214

d7de101

Removed legacy RMSE scores loaders #2214

28865fe

BernardClaxton (BernardClaxton) linked an issue Jun 25, 2026 that may be closed by this pull request

Adding more Scores diagnostics #2214

Open

BernardClaxton (BernardClaxton) self-assigned this Jun 25, 2026

BernardClaxton (BernardClaxton) added 2 commits June 25, 2026 14:05

Tests for the scores code #2214

c171e9c

Updated rose-suite.conf.example with the scores variables #2214

bb5ba3f

BernardClaxton (BernardClaxton) force-pushed the 2214_adding_scores branch from 4fb1de2 to bb5ba3f Compare June 25, 2026 13:40

BernardClaxton (BernardClaxton) requested review from David Flack (daflack) and James Frost (jfrost-mo) June 25, 2026 14:00

James Frost (jfrost-mo) changed the title ~~2214_adding_scores_pr~~ Add Scores diagnostics for spatial and timeseries RMSE, additive_bias, MAE, and Pearson's Correlation Jun 26, 2026

James Frost (jfrost-mo) changed the title ~~Add Scores diagnostics for spatial and timeseries RMSE, additive_bias, MAE, and Pearson's Correlation~~ Add Scores diagnostics for spatial and timeseries RMSE, additive_bias, MAE, and Pearson’s Correlation Coefficient Jun 26, 2026

Sylvia Bohnenstengel (Sylviabohnenstengel) self-requested a review July 14, 2026 17:48

adding conversion from iris to xarray dimension coordinates to all sc…

838bba3

…ores error metrics for spatial surface plots.

scoreswrapper typo correction.

eb11c39

Sylvia Bohnenstengel (Sylviabohnenstengel) requested changes Jul 15, 2026

View reviewed changes

BernardClaxton (BernardClaxton) added 11 commits July 15, 2026 13:14

Support for spatial case averaging in scores loaders #2214

06da2fc

Support for spatial case averaging in scores MAE recipe #2214

6a30b3c

Support for spatial case averaging in scores ME recipe #2214

08a614d

Support in scores loaders for SUBAREA_NAME #2214

f678497

Support in scores spatial MAE recipe for SUBAREA_NAME #2214

eb1240a

Support in scores spatial RMSE recipe for SUBAREA_NAME #2214

0c01120

Support in scores spatial ME recipe for SUBAREA_NAME #2214

be12c87

Support in scores timeseries MAE recipe for SUBAREA_NAME #2214

4050da6

Support in scores timeseries RMSE recipe for SUBAREA_NAME #2214

d039477

Support in scores timeseries ME recipe for SUBAREA_NAME #2214

160795b

Support in scores timeseries persons correlation recipe for SUBAREA_N…

fe1881e

…AME #2214

BernardClaxton (BernardClaxton) added 2 commits July 15, 2026 14:33

Added SPATIAL_SCORES_FIELD_METHOD into spatial scores GUI #2214

286c05f

Added SPATIAL_SCORES_FIELD_METHOD into spatial scores loaders #2214

251d5a8

BernardClaxton (BernardClaxton) and others added 6 commits July 15, 2026 14:47

Update src/CSET/loaders/verification.py

a29b37f

Co-authored-by: Sylvia Bohnenstengel <62748926+Sylviabohnenstengel@users.noreply.github.com>

Update src/CSET/operators/scoreswrappers.py

46d41a5

Co-authored-by: Sylvia Bohnenstengel <62748926+Sylviabohnenstengel@users.noreply.github.com>

Update src/CSET/recipes/verification/surface_difference_scores_additi…

6f5698e

…ve_bias.yaml Co-authored-by: Sylvia Bohnenstengel <62748926+Sylviabohnenstengel@users.noreply.github.com>

Update src/CSET/recipes/verification/surface_difference_scores_additi…

2e6afc7

…ve_bias.yaml Co-authored-by: Sylvia Bohnenstengel <62748926+Sylviabohnenstengel@users.noreply.github.com>

Merge branch 'main' into 2214_adding_scores

cdd35b0

Updated rose-suite.conf.example with SPATIAL_SCORES_FIELD_METHOD #2214

308068b

Tidied the scores metrics descriptions in rose-meta.conf #2214

08ed06c

BernardClaxton (BernardClaxton) requested a review from Sylvia Bohnenstengel (Sylviabohnenstengel) July 15, 2026 14:33

Uh oh!

Conversation

BernardClaxton (BernardClaxton) commented Jun 25, 2026 • edited by jfrost-mo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution checklist

Uh oh!

github-actions Bot commented Jun 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

BernardClaxton (BernardClaxton) commented Jun 25, 2026

Uh oh!

BernardClaxton (BernardClaxton) commented Jun 25, 2026

Uh oh!

Sylvia Bohnenstengel (Sylviabohnenstengel) commented Jul 14, 2026

Uh oh!

Sylvia Bohnenstengel (Sylviabohnenstengel) commented Jul 15, 2026

Uh oh!

Sylvia Bohnenstengel (Sylviabohnenstengel) Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sylvia Bohnenstengel (Sylviabohnenstengel) Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

BernardClaxton (BernardClaxton) commented Jul 15, 2026

Uh oh!

BernardClaxton (BernardClaxton) commented Jul 15, 2026

Uh oh!

BernardClaxton (BernardClaxton) commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

BernardClaxton (BernardClaxton) commented Jun 25, 2026 •

edited by jfrost-mo

Loading

github-actions Bot commented Jun 25, 2026 •

edited

Loading