Skip to content

Fix ContainsGroupError when writing a DataTree with region/append_dim#1036

Open
aaronspring wants to merge 1 commit into
zarr-developers:mainfrom
aaronspring:fix-datatree-region-containsgrouperror
Open

Fix ContainsGroupError when writing a DataTree with region/append_dim#1036
aaronspring wants to merge 1 commit into
zarr-developers:mainfrom
aaronspring:fix-datatree-region-containsgrouperror

Conversation

@aaronspring

Copy link
Copy Markdown

Problem

Writing a virtual DataTree with region or append_dim (forwarded to each node) always failed with zarr.errors.ContainsGroupError. virtual_datatree_to_icechunk created every node's group unconditionally with Group.from_store, but region/append_dim require the group and its arrays to already exist — so the write blew up before it ever got to writing the region/append.

This is the same class of failure as #1001, but for the DataTree path.

Fix

Mirror the branch that virtual_dataset_to_icechunk already had on main: open the existing group for region/append_dim, create it otherwise.

for store_path, vds in paths_and_virtual_datasets:
    if kwargs.get("append_dim") or kwargs.get("region"):
        group = Group.open(store=store_path, zarr_format=3)   # must already exist
    else:
        group = Group.from_store(store=store_path, zarr_format=3)  # create

Test

test_write_datatree_region was marked xfail with the reason "This doesn't work in xarray either, maybe not even intended?" — but that was a misdiagnosis. On main, running it with --runxfail fails with ContainsGroupError at the group-creation step, not in any xarray region logic. With this fix the test runs to completion and its assertions pass, so the xfail marker is removed.

🤖 Generated with Claude Code

virtual_datatree_to_icechunk unconditionally created each node's group
with Group.from_store, so forwarding region or append_dim (which require
the group and its arrays to already exist) always failed with
ContainsGroupError. Mirror the branch virtual_dataset_to_icechunk
already had: open the existing group for region/append_dim, create it
otherwise.

Removes the now-passing xfail on test_write_datatree_region, whose
stated reason (an xarray region limitation) was a misdiagnosis of this
ContainsGroupError.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.88%. Comparing base (e1ebb4b) to head (a2d7a13).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1036   +/-   ##
=======================================
  Coverage   89.87%   89.88%           
=======================================
  Files          36       36           
  Lines        2202     2204    +2     
=======================================
+ Hits         1979     1981    +2     
  Misses        223      223           
Files with missing lines Coverage Δ
virtualizarr/writers/icechunk.py 91.66% <100.00%> (+0.10%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aaronspring

Copy link
Copy Markdown
Author

CI note: the 3 red build jobs (test-py312, test-py313, minimum-versions) fail only on test_parsers/test_grib.py (test_grib_virtual_dataset, test_grib_only_variables, test_grib_chunk_decodes_via_codec) — the GRIB/gribberish parser returning empty data_vars / a NodeTypeValidationError. These are pre-existing and unrelated to this PR, which touches only the icechunk DataTree writer; all 550 other tests pass. The same 3 failures appear on #1034 (a pre-commit.ci autoupdate that changes no source), #1032, #1033, and on scheduled main runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant