Set mpi-io metadata read for file for read_access scalability

[mo-marqh]

PR Summary

Sci/Tech Reviewer:
Code Reviewer: Steve Mullerworth (@stevemullerworth)

For scalability for large rank counts of simulation ranks then it is essential that XIOS is instructed to avoid using serial netCDF calls to call nc_open on HDF5 encoded netCDF files.

XIOS uses MPI-IO collective calls for data interactions by default (good for scale) and provides an opt-in switch to use MPI-IO for metadata reads read_metadata_par read_metadata_par

But, this operates on a file object, so needs to be managed many times, or deep within the LFRic interface.

This PR proposes a full opt in of this for all uses.
Preliminary analyses show that this is not problematic at small scales and very useful at large scales.

More nuanced implementations could be explored, but do not yet seem justified, compared to this simple full opt in

Code Quality Checklist

• I have performed a self-review of my own code
• My code follows the project's style guidelines
• Comments have been included that aid understanding and enhance the readability of the code
• My changes generate no new warnings
• All automated checks in the CI pipeline have completed successfully

Testing

• I have tested this change locally, using the LFRic Core rose-stem suite
• If required (e.g. API changes) I have also run the LFRic Apps test suite using this branch
• If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
• I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
• Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Test Suite Results - lfric_core - read_metadata_par_lfric_xios/run1

Suite Information

Item	Value
Suite Name	read_metadata_par_lfric_xios/run1
Suite User	mark.hedley
Workflow Start	2026-06-29T12:49:46
Groups Run	developer

Dependency	Reference	Main Like
lfric_core	mo-marqh/lfric_core@read_metadata_par_lfric_xios	False
SimSys_Scripts	MetOffice/SimSys_Scripts@cab3315	True

Task Information

✅ succeeded tasks - 427

Test Suite Results - lfric_apps - io_trace_intercept-nc_open_par/run5

Suite Information

Item	Value
Suite Name	io_trace_intercept-nc_open_par/run5
Suite User	mark.hedley
Workflow Start	2026-06-29T08:20:19
Groups Run	lfric_atm_nwp_gal9_oper-C224_MG_ex1a_cce_production-32bit

Dependency	Reference	Main Like
casim	MetOffice/casim@2026.03.2	True
jules	MetOffice/jules@1ea67b4	True
lfric_apps	mo-marqh/lfric_apps@io_trace_intercept-nc_open_par	False
lfric_core	mo-marqh/lfric_core@metadata-mpiio-read	False
moci	MetOffice/moci@2026.03.2	True
SimSys_Scripts	MetOffice/SimSys_Scripts@3e2998b	True
socrates	MetOffice/socrates@2026.03.2	True
socrates-spectral	MetOffice/socrates-spectral@2026.03.2	True
ukca	MetOffice/ukca@1cdb9c2	True

Task Information

✅ succeeded tasks - 8

Timer Comparison

Routine	Baseline times	Branch times
lfric_atm	175.30 ≤ 175.31 ≤175.31	161.58 ≤ 161.59 ≤ 161.60
gungho_driver.initialise	27.58 ≤ 27.66 ≤ 27.67	26.26 ≤ 26.35 ≤ 26.37
gungho_driver.load_meshes	0.92 ≤ 0.98 ≤ 1.00	2.19 ≤ 2.23 ≤ 2.25
field.read	16.76 ≤ 16.82 ≤ 16.89	15.00 ≤ 15.12 ≤ 15.19
gungho_timestep	106.29 ≤ 107.37 ≤ 107.71	101.07 ≤ 102.15 ≤ 102.42
gungho_driver.finalise	28.24 ≤ 28.62 ≤ 29.00	18.32 ≤ 18.40 ≤ 18.47

Security Considerations

• I have reviewed my changes for potential security issues
• Sensitive data is properly handled (if applicable)
• Authentication and authorisation are properly implemented (if applicable)

Performance Impact

• Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

• Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

• Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

• If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team

Sci/Tech Review

• I understand this area of code and the changes being added
• The proposed changes correspond to the pull request description
• Documentation is sufficient (do documentation papers need updating)
• Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

• All dependencies have been resolved
• Related Issues have been properly linked and addressed
• CLA compliance has been confirmed
• Code quality standards have been met
• Tests are adequate and have passed
• Documentation is complete and accurate
• Security considerations have been addressed
• Performance impact is acceptable

[Steve Mullerworth (stevemullerworth)]

No issues with this. I'd be happy to Code Review if a suitably XIOS expert sci/tech review is done.

[Steve Mullerworth (stevemullerworth)]

As noted above, could I ask for a sci/tech review please.

Copilot kindly reviewed the current IO meeting notes to see if there was a firm recommendation that this should be done, but it couldn't find one. In absence of that, it would be good to have evidence of a test or a discussion within the team that has demonstrated or advised there will be a net benefit from this change.