Skip to content

cloud-radiative-effects diagnostic fails to find reference data when used with CMIP7 simulations #653

@bouweandela

Description

@bouweandela

Running

ref test-cases run --provider esmvaltool --diagnostic cloud-radiative-effects --test-case cmip7 --fetch --force-regen --clean

fails with the following error message:

...
using data sources:
- data source: LocalDataSource(name='local', project='OBS', priority=1, rootpath=PosixPath('/home/bandela/.cache/climate_ref/esmvaltool/datasets/ESMValTool/OBS'), dirname_template='Tier{tier}/{dataset}', filename_template='{project}_{dataset}_{type}_{version}_{mip}_{short_name}[_.]*nc', ignore_warnings=[])
  message: No files found matching glob pattern /home/bandela/.cache/climate_ref/esmvaltool/datasets/ESMValTool/OBS/Tier2/ISCCP-FH/OBS_ISCCP-FH_sat_v0_atmos_rsutcs[_.]*nc
2026-05-04 09:12:47,813 UTC [37859] ERROR   Set 'log_level' to 'debug' to get more information
2026-05-04 09:12:47,813 UTC [37859] ERROR   Could not create all tasks
2026-05-04 09:12:47,813 UTC [37859] ERROR   Missing data for preprocessor plot_maps/lwcre:
- Missing data for Dataset: rlut, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rlutcs, atmos, OBS, CERES-EBAF, , Ed4.2
2026-05-04 09:12:47,813 UTC [37859] ERROR   Missing data for preprocessor plot_maps/swcre:
- Missing data for Dataset: rsut, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rsutcs, atmos, OBS, CERES-EBAF, , Ed4.2
2026-05-04 09:12:47,813 UTC [37859] ERROR   Missing data for preprocessor plot_profiles/lwcre:
- Missing data for Dataset: rlut, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rlut, atmos, OBS, ESACCI-CLOUD, , AVHRR-AMPM-fv3.0
- Missing data for Dataset: rlut, atmos, OBS, ISCCP-FH, , v0
- Missing data for Dataset: rlutcs, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rlutcs, atmos, OBS, ESACCI-CLOUD, , AVHRR-AMPM-fv3.0
- Missing data for Dataset: rlutcs, atmos, OBS, ISCCP-FH, , v0
2026-05-04 09:12:47,813 UTC [37859] ERROR   Missing data for preprocessor plot_profiles/swcre:
- Missing data for Dataset: rsut, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rsut, atmos, OBS, ESACCI-CLOUD, , AVHRR-AMPM-fv3.0
- Missing data for Dataset: rsut, atmos, OBS, ISCCP-FH, , v0
- Missing data for Dataset: rsutcs, atmos, OBS, CERES-EBAF, , Ed4.2
- Missing data for Dataset: rsutcs, atmos, OBS, ESACCI-CLOUD, , AVHRR-AMPM-fv3.0
- Missing data for Dataset: rsutcs, atmos, OBS, ISCCP-FH, , v0
...

because mip is set to atmos for the CMIP5 style OBS data, while it should be Amon to match the filenames, e.g.:

/home/bandela/.cache/climate_ref/esmvaltool/datasets/ESMValTool/OBS/Tier2/ISCCP-FH/OBS_ISCCP-FH_sat_v0_Amon_rsutcs_198401-201612.nc

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions