New release seems to break open_dataset from xarray

[uriii3]

Zarr version

v3.0.9

Numcodecs version

v0.16.1

Python Version

3.12.3

Operating System

Mac

Installation

pip install

Description

From our project, we use the call of xarray.open_zarr(...) to open zarr files.

With the new release, it looks like what used to work now does not any more, with error message:
ValueError: Store is not read-only but mode is \'r\'. Unable to create a read-only copy of the store. Please use a read-only store or a storage class that implements .with_read_only()

I guess it comes from this PR but I'm not fully following everything to debug it.

Should I also pass mode='r' as an argument? Which does not seem to work.

Steps to reproduce

I will work on a MRE now, for now I have just checked that the error is indeed from the zarr version (although could come because I'm doing something wrong myself)

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "zarr@git+https://github.com/zarr-developers/zarr-python.git@main",
# ]
# ///
#
# This script automatically imports the development branch of zarr to check for issues

import zarr
# your reproducer code
# zarr.print_debug_info()

Additional output

No response

[d-v-b]

cc @TomNicholas

[TomNicholas]

IIUC the most recent release of zarr requires all stores that want to be opened by xarray to add .with_read_only. It's a really big change. See also pydata/xarray#10261 (comment)

What is the store class that's being opened here?

[uriii3]

It looks like that indeed! I actually think we have a CustomZarrStore, I will look at that, thanks!

[maxrjones]

IIUC the most recent release of zarr requires all stores that want to be opened by xarray to add .with_read_only. It's a really big change. See also pydata/xarray#10261 (comment)

the new zarr release requires that either the store's read_only attribute matches the "mode" used in zarr.open_*or .with_read_only is implemented. The most recent release of zarr does not require that all stores that want to be opened by xarray to add the .with_read_only method.

Here's an example:

The following will fail because xarray's zarr backend uses mode="r" but your store supports writes and the only current blocker for writing/deleting data is on the store class, not the array or group:

store = MyCustomZarrStore(read_only=False)
ds = xr.open_zarr(store)

This would work, regardless of whether a store implements with_read_only:

store = MyCustomZarrStore(read_only=True)
ds = xr.open_zarr(store)

I recognize this is disruptive and am sorry for the inconvenience. It seemed important to quickly release a fix for the possible unexpected data deletion associated with #3147 without the more distruptive change of blocking a group/array permissions on both Zarr's stores and external stores (e.g., #3068), but perhaps it would have been better to instead just warn the user of the possibility for data deletion (#3155) and wait for a more complete re-architecting of Zarr's permissions model.

[TomNicholas]

That's a super-helpful clarification, thanks @maxrjones.

[uriii3]

Thanks for all the quick answers and the detailed explanations! 🚀

[d-v-b]

@uriii3 glad this could be closed, and sorry for the inconvenience. I hope it wasn't too much work on your end to adapt to the changes.