Skip to content

Commit 191c7d7

Browse files
committed
Add Dataset examples to rechunk_no_shuffle notebook (#1069)
1 parent 20a090e commit 191c7d7

File tree

1 file changed

+20
-1
lines changed

1 file changed

+20
-1
lines changed

examples/user_guide/36_Rechunk_No_Shuffle.ipynb

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,6 +147,25 @@
147147
"assert result is numpy_dem\n",
148148
"print('Numpy passthrough: OK')"
149149
]
150+
},
151+
{
152+
"cell_type": "markdown",
153+
"source": "## Dataset support\n\n`rechunk_no_shuffle` also accepts `xr.Dataset`. Each dask-backed variable\nis rechunked independently; numpy-backed variables pass through unchanged.",
154+
"metadata": {}
155+
},
156+
{
157+
"cell_type": "code",
158+
"source": "ds = xr.Dataset({\n 'elevation': dem,\n 'slope': xr.DataArray(\n da.from_array(np.random.rand(4096, 4096).astype(np.float32), chunks=256),\n dims=['y', 'x'],\n ),\n 'mask': xr.DataArray(np.ones((4096, 4096), dtype=np.uint8), dims=['y', 'x']),\n})\n\nds_big = rechunk_no_shuffle(ds, target_mb=64)\n\nfor name in ds.data_vars:\n if hasattr(ds[name].data, 'dask'):\n print(f'{name}: {ds[name].chunks[0][0]} -> {ds_big[name].chunks[0][0]}')\n else:\n print(f'{name}: numpy (unchanged)')",
159+
"metadata": {},
160+
"execution_count": null,
161+
"outputs": []
162+
},
163+
{
164+
"cell_type": "code",
165+
"source": "# Works on the Dataset accessor too\nds_big_acc = ds.xrs.rechunk_no_shuffle(target_mb=64)\nassert ds_big['elevation'].chunks == ds_big_acc['elevation'].chunks\nprint('Dataset accessor: OK')",
166+
"metadata": {},
167+
"execution_count": null,
168+
"outputs": []
150169
}
151170
],
152171
"metadata": {
@@ -162,4 +181,4 @@
162181
},
163182
"nbformat": 4,
164183
"nbformat_minor": 4
165-
}
184+
}

0 commit comments

Comments
 (0)