Skip to content

Update structured Field data ingestion transposing to 4D (TZYX) and add XGrid.from_dataset() - #2091

Merged
VeckoTheGecko merged 10 commits into
v4-devfrom
4d-field-arrays
Jul 18, 2025
Merged

Update structured Field data ingestion transposing to 4D (TZYX) and add XGrid.from_dataset() #2091
VeckoTheGecko merged 10 commits into
v4-devfrom
4d-field-arrays

Conversation

@VeckoTheGecko

@VeckoTheGecko VeckoTheGecko commented Jul 17, 2025

Copy link
Copy Markdown
Contributor

Changes:

  • Add XGrid.from_dataset (rather than having to construct a xgcm grid first). This execution path drops all the Field data from the dataset before constructing the xgcm grid (and also makes it easier to call for devs as its 1 less import). Seeb73813c769aeaa92be181bf3fee1ba5275d317ce commit message for more info

  • Update Field init to transpose data to 4D array, inserting additional dummy axes where necessary

  • Chose the correct base branch (v4-dev for v4 changes)

  • Fixes Coerce StructuredGrid field data to 4D array of TZYX #2047 and works towards Testing and support for 2D fields in X-Z and Y-Z, and 1D fields in X, Y, or Z #2054

  • Added tests (for the transposing to 4D I didn't add a unit test, but added an integration test over all the generic datasets)

  • Added documentation

Will do a self review tomorrow then mark ready for review.

@VeckoTheGecko
VeckoTheGecko marked this pull request as draft July 17, 2025 17:25
@VeckoTheGecko
VeckoTheGecko force-pushed the 4d-field-arrays branch 2 times, most recently from 0da120b to 1641273 Compare July 17, 2025 17:31
@VeckoTheGecko VeckoTheGecko changed the title Update Field data ingestion transposing to 4D (TZYX) and add XGrid.from_dataset() Update structured Field data ingestion transposing to 4D (TZYX) and add XGrid.from_dataset() Jul 17, 2025
@VeckoTheGecko
VeckoTheGecko marked this pull request as ready for review July 17, 2025 17:39
Comment thread parcels/xgrid.py
Comment thread parcels/xgrid.py
Comment thread parcels/xgrid.py Outdated
Comment thread parcels/xgrid.py
Comment thread parcels/xgrid.py Outdated
Comment thread parcels/xgrid.py Outdated
Comment thread tests/utils.py Outdated
Comment thread tests/v4/test_particleset.py
Comment thread tests/v4/test_xgrid.py
Comment thread parcels/field.py
Comment thread parcels/xgrid.py
Comment thread parcels/xgrid.py Outdated
Comment thread tests/v4/test_particleset.py
Comment thread parcels/xgrid.py Outdated
Comment thread parcels/xgrid.py
Comment thread tests/v4/test_xgrid.py
Comment thread parcels/xgrid.py Outdated
VeckoTheGecko and others added 10 commits July 18, 2025 14:05
It's not necessary for us to have the Field data on the XGrid/xgcm.Grid object itself. It doesn't make a big difference in the end, but dropping the field data on grid ingestion would allow us to keep things logically separate.
Needed to update the tests because the time axis information was getting lost (due to xarray behaviour with keep_attrs).

Fixes #2047
@VeckoTheGecko
VeckoTheGecko enabled auto-merge (rebase) July 18, 2025 12:05
@VeckoTheGecko
VeckoTheGecko merged commit e576d39 into v4-dev Jul 18, 2025
8 of 9 checks passed
@VeckoTheGecko
VeckoTheGecko deleted the 4d-field-arrays branch July 18, 2025 12:10
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Parcels development Jul 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants