Skip to content

Subset gridpoint masking and distance and geographic methods for all grid types#493

Open
tlogan2000 wants to merge 20 commits into
masterfrom
subset_gridpoint_mask
Open

Subset gridpoint masking and distance and geographic methods for all grid types#493
tlogan2000 wants to merge 20 commits into
masterfrom
subset_gridpoint_mask

Conversation

@tlogan2000
Copy link
Copy Markdown
Collaborator

@tlogan2000 tlogan2000 commented Apr 30, 2026

Pull Request Checklist:

  • This PR addresses an already opened issue (for bug fixes / features)
    • This PR fixes issue #xyz
  • Tests for the changes have been added (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • HISTORY.rst has been updated (with summary of main changes)

What kind of change does this PR introduce?:

  • Adds ability to use a boolean data mask with subset_gridpoint where selected points must be within the mask (True)
  • Allows choice between using true world distance or nearest neighbour based on geographic (lat, lon) coords for selection of points for both regular and irregular grids. Current implementation is inconsistent and uses lat,lon nearest for regular grids and true distance on irregular

Does this PR introduce a breaking change?:

The default method is set for distance (more accurate but could potentially be slower for large grids). As such the default behaviour is different for regular lat, lon grids potentially causing small differences in selected locations (most likely close to poles if at all present)

Other information:

@tlogan2000 tlogan2000 requested a review from Zeitsperre April 30, 2026 15:46
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 30, 2026

Welcome, new contributor!

It appears that this is your first Pull Request. To give credit where it's due, we ask that you add your information to the AUTHORS.rst:

  • The relevant author information has been added to AUTHORS.rst

Please make sure you've read our contributing guide. We look forward to reviewing your Pull Request shortly ✨

@tlogan2000 tlogan2000 requested a review from aulemahal April 30, 2026 15:46
@tlogan2000 tlogan2000 added the enhancement New feature or request label May 4, 2026
@tlogan2000
Copy link
Copy Markdown
Collaborator Author

tlogan2000 commented May 4, 2026

@aulemahal does this potentially solve #452 as well? Haven't looked at what was done but geographic method uses a KDTree now on irregular grids

Copy link
Copy Markdown
Collaborator

@aulemahal aulemahal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Indeed, this makes #452 caduc, I had taken the liberty to simply switch to geographic, but your option to make an option is better.

A few code-related comments, but nothing to say about the logic!

Comment thread clisops/core/subset.py Outdated
Comment thread clisops/core/subset.py Outdated
Comment thread clisops/core/subset.py Outdated
Comment thread clisops/core/subset.py Outdated
Comment thread clisops/core/subset.py Outdated

else:
raise (
Exception(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Exception(
ValueError(

Not your core, but WTH is this bare Exception raising ??

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bare exceptions are bugs to be squashed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I fully understand your answer. Is my solution wanted or would you recommend something else ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this resolved?

Comment thread clisops/core/subset.py Outdated
Comment thread clisops/core/subset.py
@aulemahal
Copy link
Copy Markdown
Collaborator

@tlogan2000 I fixed it! The test simply needed to be removed, now it doesn't fail anymore.

@aulemahal
Copy link
Copy Markdown
Collaborator

aulemahal commented May 12, 2026

Ok, I was being funny. But I believe this test was superfluous.

It originates from this section of pyproj : https://github.com/pyproj4/pyproj/blob/249bfa766ff969fd73636e473e02ce20f230352c/pyproj/geod.py#L395-L407 . The _inv_point method only accepts scalars and numpy < 2 detects it and converts the size 1 arrays to floats. This is not done anymore in numpy 2 and will instead raise a TypeError. But as you can see, this is already handled by pyproj in the try-except.

Thus, the warning is not directed to us, and it's addressed anyway in the upstream code. Also, it's a DeprecationWarning, so the user won't usually see it, it was only detected because the warnings filters were modified for this test.

FYI @Zeitsperre

EDIT: For the record, we're also not required to support numpy < 2 as per the SPEC0 calendar, so in a modern environment, the warning shouldn't be raised anyway.

@Zeitsperre
Copy link
Copy Markdown
Collaborator

@aulemahal I'm planning on splitting core tools out of here and dropping numpy v2.0 at the same time. Feel free to modernize this a bit!

@aulemahal
Copy link
Copy Markdown
Collaborator

I think this PR can go without a modernization. After the drop, we could revisit everything in a more systematic way, in another PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants