PromoteLocalArrayTransformation - Add new transformation for promotion of local arrays

[wertysas]

Add PromoteLocalArrayTransformation for local array promotion in SCC pipeline

This is the first in a series of three PRs for the ecRad Loki GPU work.

This PR adds a new transformation PromoteLocalArrayTransformation that promotes local arrays with a horizontal dimension in the SCC pipeline. The transformation is included in SCCVVectorPipeline but disabled by default and can be activated with the item config key promote_locals.

PromoteLocalArrayTransformation

The transformation identifies local arrays used inside vector sections (as marked by SCCDevectorTransformation) and promotes them by appending the horizontal range dimension.
It:

• Skips arrays that already contain the horizontal dimension
• Skips arrays with entirely deferred shapes (pointers, allocatables, assumed-shape)
• By default skips arrays with only compile-time constant dimensions; this can be over
  ridden per routine via the promote_constant_arrays item config key (default False)

Changes to existing utilities

The declaration-update logic previously lived inside demote_variables but has been extracted into a shared update_variable_declarations utility in utilities.py. This is now used in both the demotion and promotion transformations. A new promote_variable_declarations in promote.py handles spec-only promotion and uses single_variable_declaration to isolate each variable before modifying shapes. A helper _is_deferred_shape decides whether promoted dimensions should also be deferred.

Furthermore, the following changes have been made:

• Dimension.size_expressions (loki/dimension.py): Now returns all lower/upper bounds pairs, not just the first bounds pair.

• SCCVVectorPipeline (loki/transformations/single_column/scc.py): PromoteLocalArrayTransformation added between SCCDemoteTransformation and SCCVecRevectorTransformation. (It is off by default but is required to sit in between the devectorization and revectorization steps).

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

[github-actions]

Documentation for this branch can be viewed at https://sites.ecmwf.int/docs/loki/682/index.html

[wertysas]

I have disabled the tests that are not compatible with OMNI. However, the tests does not seem to trigger for this PR anymore, after I force pushed the branch 5 days ago, and when I tried to manually re-run the failing test the actions was using the old cached checkout instead of the new updated tests.

[reuterbal]

Another rebase did not revive the tests but an additional commit on top seems to finally have done the trick... 🤷

[codecov]

Codecov Report

❌ Patch coverage is 98.67550% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.24%. Comparing base (bfad794) to head (633b16e).
⚠️ Report is 13 commits behind head on main.

Files with missing lines	Patch %	Lines
...nsformations/array_indexing/promote_local_array.py	95.28%	5 Missing ⚠️
loki/transformations/array_indexing/promote.py	94.73%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #682      +/-   ##
==========================================
+ Coverage   94.19%   94.24%   +0.04%     
==========================================
  Files         288      289       +1     
  Lines       47496    47965     +469     
==========================================
+ Hits        44741    45203     +462     
- Misses       2755     2762       +7     

Flag	Coverage Δ
lint_rules	96.40% <ø> (ø)
loki	94.21% <98.67%> (+0.04%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[reuterbal]

Many thanks, this looks very good but I have a few remarks how to improve the implementation - see the inline comments. In addition, there's a few things that need to be added to the tests. I've marked two inline, additionally a test that specifies preserve_arrays (ideally using non-matching upper/lower case) would be ideal.

[reuterbal]

I think we need to be careful about ignoring case here:

Suggested change

	if preserve_arrays:
	to_promote = [v for v in to_promote if v.name not in preserve_arrays]
	if preserve_arrays:
	preserve_arrays = {arr.lower() for arr in preserve_arrays}
	to_promote = [v for v in to_promote if v.name.lower() not in preserve_arrays]

[reuterbal]

I'm pretty sure this ought to become tmp(start:end, nz) to be useful

[awnawab]

I'll do a full review later today/tomorrow, but for now I can confirm that enabling PromoteLocalArrayTransformation in the sccs and sccv pipelines is safe for IFS builds.

[wertysas]

I'll do a full review later today/tomorrow, but for now I can confirm that enabling PromoteLocalArrayTransformation in the sccs and sccv pipelines is safe for IFS builds.

Great, thank you!

[awnawab]

Thanks a lot @wertysas for this very useful and well implemented new transformation 🙏 I've left a few minor comments, which don't necessarily require code changes and clarifications might be enough.

[awnawab]

Do we really need this warning? Wouldn't it be enough to just take the first lower and upper bound present in the current symbol table? And throw an error if none of them are present?

[awnawab]

Considering that the new promotion transformation is only applied to kernels, I'm wondering why this seemingly unrelated change is needed for the driver?