Skip to content

update Transmute for conv_gr_kernel_mod#582

Merged
Oakley Brunt (oakleybrunt) merged 8 commits into
MetOffice:mainfrom
MetBenjaminWent:omp_kernels_grc
Jul 15, 2026
Merged

update Transmute for conv_gr_kernel_mod#582
Oakley Brunt (oakleybrunt) merged 8 commits into
MetOffice:mainfrom
MetBenjaminWent:omp_kernels_grc

Conversation

@MetBenjaminWent

@MetBenjaminWent Benjamin Went (MetBenjaminWent) commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

PR Summary

With the existing Transmute script for the conv_gr_kernel_mod, it seems we are missing coverage, which is affecting perfomance.
image
image

The script is also very complex, and it could be simpler.

The segment loop in this kernel is extremely important, and core of the existing transmute script. If this is not parallelised with OMP, the entire call stack looses OMP threading coverage. Before these changes, it was not getting OMP coverage, which would explain the issue.

By swapping to using the kernel local script, which puts OMP around all of the i loops, the coverage has improved, and therefore the OMP performance scaling.

Before (Red) and Post (Yellow) changes:
image
I expect the 1T is the OMP overhead, but this may need further investigations.

Sci/Tech Reviewer: Hacka Fett (@christophermaynard)
Code Reviewer: Oakley Brunt (@oakleybrunt)

PR: #583
Umbrella: #424

Code Quality Checklist

  • I have performed a self-review of my own code
  • My code follows the project's style guidelines
  • Comments have been included that aid understanding and enhance the readability of the code
  • My changes generate no new warnings
  • All automated checks in the CI pipeline have completed successfully

Testing

  • I have tested this change locally, using the LFRic Apps rose-stem suite
  • If any tests fail (rose-stem or CI) the reason is understood and acceptable (e.g. kgo changes)
  • I have added tests to cover new functionality as appropriate (e.g. system tests, unit tests, etc.)
  • Any new tests have been assigned an appropriate amount of compute resource and have been allocated to an appropriate testing group (i.e. the developer tests are for jobs which use a small amount of compute resource and complete in a matter of minutes)

trac.log

Test Suite Results - lfric_apps - omp_kernels_grc/run5

Suite Information

Item Value
Suite Name omp_kernels_grc/run5
Suite User benjamin.went
Workflow Start 2026-07-02T09:26:07
Groups Run ex1a_omp_developer
Dependency Reference Main Like
casim MetOffice/casim@2026.03.2 True
jules MetOffice/jules@1ea67b4 True
lfric_apps MetBenjaminWent/lfric_apps@omp_kernels_grc False
lfric_core MetOffice/lfric_core@b85ebad True
moci MetOffice/moci@2026.03.2 True
SimSys_Scripts MetOffice/SimSys_Scripts@3e2998b True
socrates MetOffice/socrates@2026.03.2 True
socrates-spectral MetOffice/socrates-spectral@2026.03.2 True
ukca MetOffice/ukca@1cdb9c2 True

Task Information

✅ succeeded tasks - 51

Test Suite Results - lfric_apps - omp_kernels_grc/run6

Suite Information

Item Value
Suite Name omp_kernels_grc/run6
Suite User benjamin.went
Workflow Start 2026-07-02T10:19:07
Groups Run all
Dependency Reference Main Like
casim MetOffice/casim@2026.03.2 True
jules MetOffice/jules@1ea67b4 True
lfric_apps MetBenjaminWent/lfric_apps@omp_kernels_grc False
lfric_core MetOffice/lfric_core@b85ebad True
moci MetOffice/moci@2026.03.2 True
SimSys_Scripts MetOffice/SimSys_Scripts@3e2998b True
socrates MetOffice/socrates@2026.03.2 True
socrates-spectral MetOffice/socrates-spectral@2026.03.2 True
ukca MetOffice/ukca@1cdb9c2 True

Task Information

❌ failed tasks - 1
Task State
run_shallow_water_galewsky_vi_ffsl-C96_ex1a_gnu_fast-debug-64bit failed
✅ succeeded tasks - 1590

Test Suite Results - lfric_apps - omp_kernels_grc/run7

Suite Information

Item Value
Suite Name omp_kernels_grc/run7
Suite User benjamin.went
Workflow Start 2026-07-02T12:12:38
Groups Run ex1a_omp_C192_cce_production
Dependency Reference Main Like
casim MetOffice/casim@2026.03.2 True
jules MetOffice/jules@1ea67b4 True
lfric_apps MetBenjaminWent/lfric_apps@omp_kernels_grc False
lfric_core MetOffice/lfric_core@b85ebad True
moci MetOffice/moci@2026.03.2 True
SimSys_Scripts MetOffice/SimSys_Scripts@3e2998b True
socrates MetOffice/socrates@2026.03.2 True
socrates-spectral MetOffice/socrates-spectral@2026.03.2 True
ukca MetOffice/ukca@1cdb9c2 True

Task Information

✅ succeeded tasks - 11

Security Considerations

  • I have reviewed my changes for potential security issues
  • Sensitive data is properly handled (if applicable)
  • Authentication and authorisation are properly implemented (if applicable)

Performance Impact

  • Performance of the code has been considered and, if applicable, suitable performance measurements have been conducted

AI Assistance and Attribution

  • Some of the content of this change has been produced with the assistance of Generative AI tool name (e.g., Met Office Github Copilot Enterprise, Github Copilot Personal, ChatGPT GPT-4, etc) and I have followed the Simulation Systems AI policy (including attribution labels)

Documentation

  • Where appropriate I have updated documentation related to this change and confirmed that it builds correctly

PSyclone Approval

  • If you have edited any PSyclone-related code (e.g. PSyKAl-lite, Kernel interface, optimisation scripts, LFRic data structure code) then please contact the TCD Team

Sci/Tech Review

  • I understand this area of code and the changes being added
  • The proposed changes correspond to the pull request description
  • Documentation is sufficient (do documentation papers need updating)
  • Sufficient testing has been completed

(Please alert the code reviewer via a tag when you have approved the SR)

Code Review

  • All dependencies have been resolved
  • Related Issues have been properly linked and addressed
  • CLA compliance has been confirmed
  • Code quality standards have been met
  • Tests are adequate and have passed
  • Documentation is complete and accurate
  • Security considerations have been addressed
  • Performance impact is acceptable

@MetBenjaminWent

Copy link
Copy Markdown
Contributor Author

Segments don't appear to need OMP across (existing script):
image

@MetBenjaminWent

Copy link
Copy Markdown
Contributor Author

Segments don't appear to need OMP across (existing script): image

The loop here is very important, and core of the existing transmute script. If this is not parallelised with OMP, the entire call stack looses OMP threading coverage.......

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Ben :D

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes simplify the transmute scripts and crucially add OpenMP around the main
segmented loop over the domain. Previously, this was missing and so the main loop wasn't parallel. It is with these changes. Approved.

Benjamin Went (MetBenjaminWent) added a commit to MetBenjaminWent/lfric_apps that referenced this pull request Jul 10, 2026
@MetBenjaminWent

Copy link
Copy Markdown
Contributor Author

Since move to Psyclone 3.3.1, we need fixes for this PR in #607 (which also contains this PR).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CR approved

@MetBenjaminWent

Copy link
Copy Markdown
Contributor Author

Note, work in #607 has highlighted that the options list no longer works for ignore_dependencies_for for PSyclone 3.3.1, so this script would continue to not effectively function!

I've pulled in the fixes from #607, so that this will now work with Psyclone 3.3.1

Benjamin Went (MetBenjaminWent) added a commit to MetBenjaminWent/lfric_apps that referenced this pull request Jul 14, 2026
@oakleybrunt Oakley Brunt (oakleybrunt) merged commit 987533b into MetOffice:main Jul 15, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants