Skip to content

Transmute Kernels to add OMP where not already - fix existing functionality#607

Draft
Benjamin Went (MetBenjaminWent) wants to merge 39 commits into
MetOffice:mainfrom
MetBenjaminWent:add_omp_more_kernels
Draft

Transmute Kernels to add OMP where not already - fix existing functionality#607
Benjamin Went (MetBenjaminWent) wants to merge 39 commits into
MetOffice:mainfrom
MetBenjaminWent:add_omp_more_kernels

Conversation

@MetBenjaminWent

@MetBenjaminWent Benjamin Went (MetBenjaminWent) commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

PR Summary

As noted in the umbrella issue, not all of the kernels have OMP coverage, which means we are loosing some performance at higher thread counts, even if it's smaller a smaller runtime cost than some other sections.
Also with the addition of a local script for kernels, we need to move the SCRIPT_OPTIONS_DICT for mphys_kernel_mod from the global area to the kernels.
With the move to the head of PSyclone, 3.3.1, it seems that the options parameter passed to a transformation, no longer passes everything through, such as ignore_dependencies_for. I am aware that STFC are trying to deprecate the options dict in favor of explicit by optional parameters to all transformations. We can catch modifications in this PR.
As I'm already fixing some piping in the scripts, where sensible there is some housekeeping, and lifting duplicate scripts into the local script.

This PR Contains:

  • OMP additions for the following kernels:
  • Move SCRIPT_OPTIONS_DICT for mphys_kernel_mod to kernel - Moved up into Enable Transmute script for bl imp and exp kernels #361
  • Update kernel local.py script for PSyclone 3.3.1. - Part moved up into Enable Transmute script for bl imp and exp kernels #361, part here.
  • Update global.py script for PSyclone 3.3.1.
  • Update pc2_conv_coupling_kernel_mod.py for PSyclone 3.3.1.
  • Update lw_kernel_mod and sw_kernel_mod for PSyclone 3.3.1.
  • Update conv_gr_kernel_mod for PSyclone 3.3.1. - Moved into update Transmute for conv_gr_kernel_mod #583
  • Lift bm_tau_kernel_mod to use the local.py script as scripts are effectively identical.
  • Lift jules_extra_kernel_mod to use the local.py script as scripts are effectively identical.
  • Lift pc2_initiation_kernel_mod to use the local.py script as scripts are effectively identical.
  • Lift sw_rad_tile_kernel_mod to use the local.py script as scripts are effectively identical.
  • Lift jules_exp_kernel_mod to use the local.py script. Add checks for OMP ancestory for general safety, and option of pure call overrides to local.py script.
  • Lift jules_imp_kernel_mod to use the local.py script as scripts are effectively identical.

CCE Issues currently with, but have still been captured in this ticket to be ready to be activated as needed :
aerosol_ukca_kernel_mod, conv_comorph_kernel_mod

Sci/Tech Reviewer:
Code Reviewer: Joseph Abram (@J-J-Abram)

Issue: #608
Umbrella: #424

Code Quality Checklist

  • I have performed a self-review of my own code
  • My code follows the project's style guidelines
  • Comments have been included that aid understanding and enhance the readability of the code
  • My changes generate no new warnings
  • All automated checks in the CI pipeline have completed successfully

Testing

  • I have tested this change locally, using the LFRic Apps rose-stem suite
  • If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
  • I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
  • Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Test Suite Results - lfric_apps - add_omp_more_kernels/run3

Suite Information

Item Value
Suite Name add_omp_more_kernels/run3
Suite User benjamin.went
Workflow Start 2026-07-13T15:24:34
Groups Run ex1a_omp_developer
Dependency Reference Main Like
casim MetOffice/casim@2026.07.1 True
jules MetOffice/jules@2026.07.1 True
lfric_apps MetBenjaminWent/lfric_apps@add_omp_more_kernels False
lfric_core MetOffice/lfric_core@2026.07.1 True
moci MetOffice/moci@2026.07.1 True
SimSys_Scripts MetOffice/SimSys_Scripts@2026.07.1 True
socrates MetOffice/socrates@2026.07.1 True
socrates-spectral MetOffice/socrates-spectral@2026.07.1 True
ukca MetOffice/ukca@2026.07.1 True

Task Information

✅ succeeded tasks - 51

Test Suite Results - lfric_apps - add_omp_more_kernels/run4

Suite Information

Item Value
Suite Name add_omp_more_kernels/run4
Suite User benjamin.went
Workflow Start 2026-07-14T10:22:42
Groups Run all
Dependency Reference Main Like
casim MetOffice/casim@2026.07.1 True
jules MetOffice/jules@2026.07.1 True
lfric_apps MetBenjaminWent/lfric_apps@add_omp_more_kernels False
lfric_core MetOffice/lfric_core@2026.07.1 True
moci MetOffice/moci@2026.07.1 True
SimSys_Scripts MetOffice/SimSys_Scripts@2026.07.1 True
socrates MetOffice/socrates@2026.07.1 True
socrates-spectral MetOffice/socrates-spectral@2026.07.1 True
ukca MetOffice/ukca@2026.07.1 True

Task Information

✅ succeeded tasks - 1592

Security Considerations

  • I have reviewed my changes for potential security issues
  • Sensitive data is properly handled (if applicable)
  • Authentication and authorisation are properly implemented (if applicable)

Performance Impact

  • Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

  • Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

  • Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

  • If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team

Sci/Tech Review

  • I understand this area of code and the changes being added
  • The proposed changes correspond to the pull request description
  • Documentation is sufficient (do documentation papers need updating)
  • Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

  • All dependencies have been resolved
  • Related Issues have been properly linked and addressed
  • CLA compliance has been confirmed
  • Code quality standards have been met
  • Tests are adequate and have passed
  • Documentation is complete and accurate
  • Security considerations have been addressed
  • Performance impact is acceptable

@MetBenjaminWent

Copy link
Copy Markdown
Contributor Author

Some files are running into this error:
Transformation Error: Loop cannot be parallelised because psyclone cannot guarantee that the accesses to [item] are arrays or pure calls. If they are but the symbol is imported, try adding the module name to RESOLVE_IMPORTS.

#613 has been raised to look at this again when the build system can accommodate it.

@MetBenjaminWent

Benjamin Went (MetBenjaminWent) commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

Issue with CCE build time is cropping up sadly.

The problem was a bug/feature in psyclone where it changed a case statement into nested if statements instead of using if-else-if, which really upsets the compiler. Both the files you've mentioned have giant case statements, so I think you're probably right.

Work here for aerosol_ukca_kernel_mod and conv_comorph_kernel_mod is being paused.

Fix to PSyclone in their Issue here: stfc/PSyclone#3418

Benjamin Went (MetBenjaminWent) added a commit to MetBenjaminWent/lfric_apps that referenced this pull request Jul 14, 2026

@TeranIvy Iva Kavcic (TeranIvy) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the heads-up on approval/review, Benjamin Went (@MetBenjaminWent). I have two requests/questions in the modified transformation scripts (see inline):

  • There is no year in the copyright in some of the scripts, and in some is. What is the standard on this?
  • The input argument psyir and the return/result of the function trans are not documented, and it would be useful for the understanding of functionality.

Comment thread applications/lfric_atm/optimisation/meto-ex1a/transmute/kernel/local.py Outdated
Comment thread applications/lfric_atm/optimisation/meto-ex1a/transmute/kernel/script_options.py Outdated
Comment thread applications/lfric_atm/optimisation/meto-ex1a/transmute/global.py Outdated

@TeranIvy Iva Kavcic (TeranIvy) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making changes. I'm afraid I misunderstood documentation of return in the trans functions. PSyclone convention only does it in internal classes/functions. I see they're empty in your changes so I'll just ask for the deletion of empty returns. Otherwise it's all well.

@TeranIvy Iva Kavcic (TeranIvy) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants