Skip to content

Commit 84f47d3

Browse files
committed
doc notes about this being experimental until 3.3
1 parent 9024817 commit 84f47d3

7 files changed

Lines changed: 28 additions & 7 deletions

File tree

changes/3534.feature.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
11
Adds support for `RectilinearChunkGrid`, enabling arrays with variable chunk sizes along each dimension in Zarr v3. Users can now specify irregular chunking patterns using nested sequences: `chunks=[[10, 20, 30], [25, 25, 25, 25]]` creates an array with 3 chunks of sizes 10, 20, and 30 along the first dimension, and 4 chunks of size 25 along the second dimension. This feature is useful for data with non-uniform structure or when aligning chunks with existing data partitions. Note that `RectilinearChunkGrid` is only supported in Zarr format 3 and cannot be used with sharding or when creating arrays from existing data via `from_array()`.
2+
3+
Note: `RectilinearChunkGrid` is an experimental feature and may change in future releases. This feature is expected to stabilize in version 3.3.

docs/user-guide/arrays.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -568,6 +568,10 @@ Without the `shards` argument, there would be 10,000 chunks stored as individual
568568

569569
## Variable Chunking (Zarr v3)
570570

571+
!!! warning "Experimental Feature"
572+
Variable chunking with `RectilinearChunkGrid` is an experimental feature and may change
573+
in future releases. This feature is expected to stabilize in Zarr version 3.3.
574+
571575
In addition to regular chunking where all chunks have the same size, Zarr v3 supports
572576
**variable chunking** (also called rectilinear chunking), where chunks can have different
573577
sizes along each dimension. This is useful when your data has non-uniform structure or

src/zarr/api/asynchronous.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -929,7 +929,8 @@ async def create(
929929
Chunk shape. If True, will be guessed from ``shape`` and ``dtype``. If
930930
False, will be set to ``shape``, i.e., single chunk for the whole array.
931931
If an int, the chunk size in each dimension will be given by the value
932-
of ``chunks``. Default is True.
932+
of ``chunks``. Default is True. Also supports nested sequences for
933+
variable-sized chunks (Zarr format 3 only, experimental until 3.3).
933934
dtype : str or dtype, optional
934935
NumPy dtype.
935936
compressor : Codec, optional

src/zarr/api/synchronous.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -665,7 +665,8 @@ def create(
665665
Chunk shape. If True, will be guessed from ``shape`` and ``dtype``. If
666666
False, will be set to ``shape``, i.e., single chunk for the whole array.
667667
If an int, the chunk size in each dimension will be given by the value
668-
of ``chunks``. Default is True.
668+
of ``chunks``. Default is True. Also supports nested sequences for
669+
variable-sized chunks (Zarr format 3 only, experimental until 3.3).
669670
dtype : str or dtype, optional
670671
NumPy dtype.
671672
compressor : Codec, optional
@@ -863,7 +864,8 @@ def create_array(
863864
Chunk shape of the array. Several formats are supported:
864865
865866
- tuple of ints: Creates a RegularChunkGrid with uniform chunks, e.g., ``(10, 10)``
866-
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks (Zarr format 3 only),
867+
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks
868+
(Zarr format 3 only, experimental until 3.3),
867869
e.g., ``[[10, 20, 30], [5, 5]]`` creates variable chunks along each dimension
868870
- ChunkGrid instance: Uses the provided chunk grid directly (Zarr format 3 only)
869871
- "auto": Automatically determines chunk shape based on array shape and dtype

src/zarr/core/array.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4986,7 +4986,8 @@ async def create_array(
49864986
Chunk shape of the array. Several formats are supported:
49874987
49884988
- tuple of ints: Creates a RegularChunkGrid with uniform chunks, e.g., ``(10, 10)``
4989-
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks (Zarr format 3 only),
4989+
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks
4990+
(Zarr format 3 only, experimental until 3.3),
49904991
e.g., ``[[10, 20, 30], [5, 5]]`` creates variable chunks along each dimension
49914992
- ChunkGrid instance: Uses the provided chunk grid directly (Zarr format 3 only)
49924993
- "auto": Automatically determines chunk shape based on array shape and dtype

src/zarr/core/chunk_grids.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -195,6 +195,10 @@ class RectilinearChunks(ChunksType):
195195
"""
196196
Chunk specification for arrays with variable chunk sizes per dimension.
197197
198+
.. warning::
199+
This is an experimental feature and may change in future releases.
200+
Expected to stabilize in Zarr version 3.3.
201+
198202
Behaves like a tuple of tuples, where each inner tuple contains the
199203
chunk sizes along that dimension. Provides named access and additional
200204
methods for introspection when dimension_names are available.
@@ -866,6 +870,10 @@ class RectilinearChunkGrid(ChunkGrid):
866870
"""
867871
A rectilinear chunk grid where chunk sizes vary along each axis.
868872
873+
.. warning::
874+
This is an experimental feature and may change in future releases.
875+
Expected to stabilize in Zarr version 3.3.
876+
869877
Attributes
870878
----------
871879
chunk_shapes : tuple[tuple[int, ...], ...]

src/zarr/core/group.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1056,7 +1056,8 @@ async def create_array(
10561056
Chunk shape of the array. Several formats are supported:
10571057
10581058
- tuple of ints: Creates a RegularChunkGrid with uniform chunks, e.g., ``(10, 10)``
1059-
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks (Zarr format 3 only),
1059+
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks
1060+
(Zarr format 3 only, experimental until 3.3),
10601061
e.g., ``[[10, 20, 30], [5, 5]]`` creates variable chunks along each dimension
10611062
- ChunkGrid instance: Uses the provided chunk grid directly (Zarr format 3 only)
10621063
- "auto": Automatically determines chunk shape based on array shape and dtype
@@ -2490,7 +2491,8 @@ def create(
24902491
Chunk shape of the array. Several formats are supported:
24912492
24922493
- tuple of ints: Creates a RegularChunkGrid with uniform chunks, e.g., ``(10, 10)``
2493-
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks (Zarr format 3 only),
2494+
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks
2495+
(Zarr format 3 only, experimental until 3.3),
24942496
e.g., ``[[10, 20, 30], [5, 5]]`` creates variable chunks along each dimension
24952497
- ChunkGrid instance: Uses the provided chunk grid directly (Zarr format 3 only)
24962498
- "auto": Automatically determines chunk shape based on array shape and dtype
@@ -2639,7 +2641,8 @@ def create_array(
26392641
Chunk shape of the array. Several formats are supported:
26402642
26412643
- tuple of ints: Creates a RegularChunkGrid with uniform chunks, e.g., ``(10, 10)``
2642-
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks (Zarr format 3 only),
2644+
- nested sequence: Creates a RectilinearChunkGrid with variable-sized chunks
2645+
(Zarr format 3 only, experimental until 3.3),
26432646
e.g., ``[[10, 20, 30], [5, 5]]`` creates variable chunks along each dimension
26442647
- ChunkGrid instance: Uses the provided chunk grid directly (Zarr format 3 only)
26452648
- "auto": Automatically determines chunk shape based on array shape and dtype

0 commit comments

Comments
 (0)