Skip to content

Add complex dtype support to FillValueCoder for Zarr backend#11151

Merged
jsignell merged 12 commits intopydata:mainfrom
maxrjones:complex_FillValue
Apr 9, 2026
Merged

Add complex dtype support to FillValueCoder for Zarr backend#11151
jsignell merged 12 commits intopydata:mainfrom
maxrjones:complex_FillValue

Conversation

@maxrjones
Copy link
Copy Markdown
Contributor

This PR adds support for encoding/decoding complex fill values (complex64, complex128) in the Zarr backend's FillValueCoder, which previously raised ValueError for complex dtypes. This enables converting/virtualizing NetCDF files with these fill values to Zarr. As an example, this change unlocks virtualizing full NISAR granules using VirtualiZarr + Xarray.

Each component (real/imag) is base64-encoded as a little-endian double, consistent with how float fill values are already handled and ensuring safe JSON round-tripping of special values like NaN and Infinity.

  • Closes #xxxx
  • Tests added
  • User visible changes (including notable bug fixes) are documented in whats-new.rst
  • New functions/methods are listed in api.rst

@github-actions github-actions bot added topic-backends topic-zarr Related to zarr storage library io labels Feb 9, 2026
@maxrjones
Copy link
Copy Markdown
Contributor Author

@dcherian thanks for your review. Do I need to get mypy working for this PR to go into main? The failures are in the function that I worked on, but unrelated to the changes that I made.

Copy link
Copy Markdown
Member

@jsignell jsignell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Mentioned offline that I think the suggestion in https://github.com/pydata/xarray/pull/11151/changes#r2805178961 was to write a hypothesis test, so just that and one little nit.

@github-actions github-actions bot added the topic-hypothesis Strategies or tests using the hypothesis library label Apr 9, 2026
@jsignell jsignell added the plan to merge Final call for comments label Apr 9, 2026
@jsignell
Copy link
Copy Markdown
Member

jsignell commented Apr 9, 2026

@maxrjones the hypothesis test looks good. Whenever you are done, just add a line to "what's new" and then this is good to merge.

@maxrjones
Copy link
Copy Markdown
Contributor Author

@maxrjones the hypothesis test looks good. Whenever you are done, just add a line to "what's new" and then this is good to merge.

thanks, Julia! I added this as a "feature", but wasn't 100% confident in the categorization. Please let me know if I should move it elsewhere.

@jsignell jsignell merged commit 6155c47 into pydata:main Apr 9, 2026
41 checks passed
spencerkclark pushed a commit to spencerkclark/xarray that referenced this pull request Apr 11, 2026
…11151)

Add support for encoding/decoding complex fill values (complex64, complex128) in the Zarr backend's FillValueCoder, which previously raised ValueError for complex dtypes. This enables converting/virtualizing NetCDF files with these fill values to Zarr.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

io plan to merge Final call for comments topic-backends topic-hypothesis Strategies or tests using the hypothesis library topic-zarr Related to zarr storage library

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants