Skip to content

Commit 3aa969d

Browse files
committed
docs(cluster): πŸ“ trim snakemake.md to SLURM/cluster-specific content
Custom-function contract/examples, measurements, and label-relations content now live on their own pages (previous commit) β€” replace the inline copies here with links, and fix every cross-reference elsewhere (dog.md, ome_zarr_napari.md, dog.py docstring) that pointed at the old anchors.
1 parent 7914458 commit 3aa969d

4 files changed

Lines changed: 24 additions & 228 deletions

File tree

β€Ždocs/examples/dog.mdβ€Ž

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -97,12 +97,12 @@ tile_process(IMAGE, fn, tile_shape=(1, 1024, 1024), overlap=8, write_to=OUTPUT)
9797

9898
On the cluster, set `dilate: 2` in the YAML config instead β€” it applies to
9999
`method: "custom"` (this plugin) the same way it does for `cellpose`/
100-
`threshold`, see [Growing labels after segmentation](../guide/snakemake.md#growing-labels-afterwards-dilation).
100+
`threshold`, see [Growing labels afterwards](../guide/custom_segmentation.md#growing-labels-afterwards-dilation).
101101

102102
## Using it in the Snakemake workflow
103103

104104
No dedicated wiring needed β€” `patchworks.plugins.dog` exposes a `segment(tile, **kwargs)`
105-
adapter for the documented [`"custom"` method](../guide/snakemake.md#custom-segmentation-function):
105+
adapter for the documented [`"custom"` method](../guide/custom_segmentation.md):
106106

107107
```yaml
108108
method: "custom"
@@ -185,6 +185,6 @@ Checklist specific to this config:
185185

186186
Segment the cell body with Cellpose and the cilia with `dog_label_fn` as two
187187
separate `tile_process` runs (same image, same `tile_shape`), then use
188-
[`label_relations`](../guide/snakemake.md#relating-labels-across-segmentations)
188+
[`label_relations`](../guide/label_relations.md)
189189
to map each cilium to the cell it belongs to β€” see
190190
`workflow/config/multi.yaml` for the same thing wired up as a cluster job.

β€Ždocs/guide/ome_zarr_napari.mdβ€Ž

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ just one channel instead: `view_in_napari("scan.zarr", channel=0)`.
159159
out-of-core straight off the Labels layer's backing dask/zarr array, so
160160
it scales to the same huge label images `tile_process` writes, unlike
161161
plain `skimage.measure.regionprops`. Bundled in `patchworks[napari]`. See
162-
[Measurements](snakemake.md#measurements-fast-whole-volume-regionprops)
162+
[Measurements](measurements.md)
163163
for the non-interactive/headless equivalent.
164164

165165
## End-to-end

β€Ždocs/guide/snakemake.mdβ€Ž

Lines changed: 19 additions & 223 deletions
Original file line numberDiff line numberDiff line change
@@ -80,15 +80,9 @@ sequential_labels: true # renumber labels to a contiguous 1..N
8080
8181
!!! tip "Growing labels after segmentation"
8282
`dilate: N` grows every label by `N` pixels once segmentation finishes,
83-
regardless of `method` (`cellpose`, `threshold`, or `custom`). It runs
84-
per-tile, before the overlap halo is trimmed and tiles are merged, so
85-
dilated labels still stitch correctly across tile boundaries β€” just make
86-
sure `overlap` covers the dilation amount plus the usual object-diameter
87-
halo. `0` (default) disables it. Under the hood this wraps whatever
88-
segmentation function `method` builds with
89-
[`patchworks.dilate_labels`](../api/postprocess.md); see [Custom
90-
segmentation function](#custom-segmentation-function) below for using it
91-
directly from Python instead of via config.
83+
regardless of `method`. `0` (default) disables it. See [Growing labels
84+
afterwards](custom_segmentation.md#growing-labels-afterwards-dilation)
85+
for how it works and the equivalent direct-API call.
9286

9387
!!! tip "Tile size vs runtime"
9488
`tile_shape: "auto"` sizes each tile to your GPU's VRAM. Smaller tiles =
@@ -266,39 +260,11 @@ results/image.zarr/labels/cyto_labels/
266260
Different segmentations of the same image can use different `channel` and
267261
`cellpose:` settings freely, but keep `tile_shape`/`level` the same across
268262
configs β€” the label arrays then share the exact same chunk layout, which
269-
`label_relations()` (below) requires.
263+
[`label_relations()`](label_relations.md) requires.
270264

271-
### Relating labels across segmentations
272-
273-
Once you have two segmentations of the same image (e.g. nuclei inside cells),
274-
`label_relations()` maps each label in one to the label it overlaps most in
275-
the other β€” by streaming both arrays chunk by chunk, so it scales to
276-
hundreds of thousands of objects without loading anything fully into RAM:
277-
278-
```python
279-
import dask.array as da
280-
from patchworks import label_relations
281-
282-
nuclei = da.from_zarr("results/image.zarr", component="labels/nuclei_labels/0")
283-
cells = da.from_zarr("results/image.zarr", component="labels/cyto_labels/0")
284-
285-
table = label_relations(nuclei, cells)
286-
table[2]
287-
# {'match': 3, 'overlap_voxels': 4821, 'overlap_fraction': 0.94}
288-
# -> nucleus 2 belongs to cell 3, 94% of its voxels fall inside it
289-
```
290-
291-
Save it as a table:
292-
293-
```python
294-
import csv
295-
296-
with open("nuclei_to_cell.csv", "w", newline="") as f:
297-
w = csv.writer(f)
298-
w.writerow(["nucleus_id", "cell_id", "overlap_voxels", "overlap_fraction"])
299-
for nucleus_id, m in table.items():
300-
w.writerow([nucleus_id, m["match"], m["overlap_voxels"], m["overlap_fraction"]])
301-
```
265+
See [Relating labels across segmentations](label_relations.md) for what
266+
`label_relations()` returns and how to save it yourself β€” the cluster
267+
workflow's own automation is below.
302268

303269
### One command: multiple segmentations + relations
304270

@@ -338,9 +304,9 @@ extra) with two sheets:
338304
| `<a>` | every non-background `a` label, **including unmatched ones** | `<a>_id`, `<b>_id` (blank if unmatched), `overlap_voxels`, `overlap_fraction` (0 if unmatched) |
339305
| `<b>` | every non-background `b` label, **including ones with zero matches** | `<b>_id`, `<a>_count`, `total_overlap_voxels` |
340306

341-
Unlike calling `label_relations()` directly (which only returns matched `a`
342-
labels β€” see below), the workbook always covers every object in both
343-
segmentations, so counts (e.g. "how many nuclei have no matching cell",
307+
Unlike calling [`label_relations()`](label_relations.md) directly (which
308+
only returns matched `a` labels), the workbook always covers every object in
309+
both segmentations, so counts (e.g. "how many nuclei have no matching cell",
344310
"how many cells have zero cilia") aren't silently dropped.
345311

346312
Both lists are ordinary lists, so 3+ segmentations work the same way β€” add
@@ -358,189 +324,19 @@ cyto_labels` and `cilia_labels -> nuclei_labels`) so you can use whichever
358324
fits a given dataset. See `config/config_cilia.yaml`. Its deconvolution step
359325
needs `pip install "patchworks[dog]"` in the segment jobs' environment.
360326

361-
## Measurements (fast, whole-volume regionprops)
362-
363-
`skimage.measure.regionprops` needs the full labelled + intensity array in
364-
RAM β€” fine for one tile, not for a hundred-thousand-object OME-ZARR.
365-
366-
**Interactively, in napari**, this is what
367-
[napari-chunked-regionprops](https://github.com/imcf/napari-chunked-regionprops)
368-
is for β€” its "Measure" dock widget computes area/centroid/intensity stats
369-
directly off a Labels layer's dask/zarr-backed array, out-of-core, and scales
370-
with chunk count rather than object count. It's the best fit for measuring
371-
*every* object in a store this size, not just a cropped region β€” see
372-
[View image + labels in napari](ome_zarr_napari.md#view-image--labels-in-napari).
373-
Bundled in `patchworks[napari]`.
327+
## Measurements
374328

375-
**Headless/scripted**, use [`dask-image`](https://image.dask.org)'s
376-
`ndmeasure`, which computes directly on the dask/zarr-backed arrays,
377-
chunk-parallel, without materializing the volume:
378-
379-
```bash
380-
pip install dask-image
381-
```
382-
383-
```python
384-
import dask.array as da
385-
from dask_image.ndmeasure import area, center_of_mass, mean, standard_deviation
386-
387-
labels = da.from_zarr("results/image.zarr", component="labels/cyto_labels/0")
388-
image = da.from_zarr("results/image.zarr", component="0")[0] # channel 0, level 0
389-
390-
ids = da.unique(labels[labels > 0]).compute()
391-
areas = area(image, labels, ids).compute() # voxel counts
392-
means = mean(image, labels, ids).compute() # mean intensity
393-
stds = standard_deviation(image, labels, ids).compute()
394-
centroids = center_of_mass(image, labels, ids).compute() # voxel coords (z, y, x)
395-
```
396-
397-
Multiply `areas` by the voxel's physical volume and `centroids` by the pixel
398-
size (both read straight from the OME-ZARR's own `multiscales` metadata) to
399-
get Β΅m-scale measurements.
400-
401-
For interactively inspecting individual cells by clicking in the viewer
402-
(not all objects at once), the
403-
[napari-skimage-regionprops](https://github.com/haesleinhuepf/napari-skimage-regionprops)
404-
plugin's table widget works well β€” point it at a cropped region rather than
405-
the full volume, since it loads its input fully into memory. For measuring
406-
*all* objects at once, use
407-
[napari-chunked-regionprops](https://github.com/imcf/napari-chunked-regionprops)
408-
instead (see above) β€” it doesn't have this in-memory limitation.
329+
See [Measurements](measurements.md) for computing area/centroid/intensity
330+
stats on a whole label store, interactively in napari or headless/scripted β€”
331+
`skimage.measure.regionprops` alone doesn't scale to a store this size.
409332

410333
## Custom segmentation function
411334

412-
Not using Cellpose? Run **your own** per-tile function β€” no need to edit the
413-
package. You write one function; patchworks handles everything around it
414-
(tiling, halos, skipping empty tiles, the zarr-native merge, global relabelling,
415-
resume, logs).
416-
417-
### The contract
418-
419-
Your function is called **once per tile**:
420-
421-
```python
422-
labels = segment(tile) # plus any kwargs you configure
423-
```
424-
425-
| | What you get / must return |
426-
| --- | --- |
427-
| **Input `tile`** | A NumPy array of **one** tile, with the overlap halo already included. The channel and pyramid level from the config are already selected, so it is purely spatial: `(z, y, x)` for a 3-D run, `(y, x)` for 2-D. Dtype is the image's (e.g. `uint16`). |
428-
| **Return** | An integer **label** array (not a boolean mask), **same shape** as `tile`. `0` = background; each object a distinct positive integer. |
429-
| **Labels** | Only need to be unique **within the tile**. Don't try to make them globally unique β€” the merge step stitches objects across tile borders and renumbers everything to a contiguous `1..N` (`sequential_labels: true`). |
430-
| **Shape** | Must match the input exactly β€” patchworks trims the halo off your output, so a wrong shape is an error. Don't crop or resize inside the function. |
431-
432-
That is the whole interface. Anything that turns an image tile into a label
433-
image works: classic image processing, StarDist, a trained model, an external
434-
binary you shell out to, …
435-
436-
### Minimal example (no GPU, no deps beyond scikit-image)
437-
438-
```python
439-
# my_seg.py
440-
import numpy as np
441-
from skimage.measure import label
442-
443-
def segment(tile: np.ndarray, sigma: float = 2.0) -> np.ndarray:
444-
"""Threshold + connected components. Returns int32 labels (0 = bg)."""
445-
from skimage.filters import gaussian, threshold_otsu
446-
447-
smooth = gaussian(tile, sigma=sigma, preserve_range=True)
448-
thr = threshold_otsu(smooth) if smooth.max() > smooth.min() else np.inf
449-
return label(smooth > thr).astype("int32")
450-
```
451-
452-
```yaml
453-
method: "custom"
454-
label_name: "my_labels"
455-
custom:
456-
module: "my_seg" # import name (see "Make it importable")
457-
function: "segment" # default is "segment"
458-
kwargs: # optional β€” forwarded as segment(tile, **kwargs)
459-
sigma: 1.5
460-
```
461-
462-
### Growing labels afterwards (dilation)
463-
464-
To grow every label by a few pixels after segmentation β€” any method, not
465-
just `custom` β€” set `dilate: N` in the config (see the tip above), or wrap
466-
your function directly with
467-
[`patchworks.dilate_labels`](../api/postprocess.md) when calling the API
468-
yourself:
469-
470-
```python
471-
from patchworks import tile_process, dilate_labels
472-
from patchworks.plugins.dog import dog_label_fn
473-
474-
fn = dog_label_fn(low_sigma=1.0, high_sigma=3.0, threshold=0.02)
475-
fn = dilate_labels(fn, iterations=2) # grow each label by 2 px, then run
476-
result = tile_process("image.zarr", fn, tile_shape=(1, 2048, 2048),
477-
overlap=8, write_to="labels.zarr")
478-
```
479-
480-
`dilate_labels` wraps any `(tile) -> labels` function β€” the same contract
481-
described above β€” so it works with `dog_label_fn`, `cellpose_fn`, or your
482-
own `segment`. It dilates each tile's labels before the halo is trimmed and
483-
tiles are merged, so `overlap` must still cover the dilation amount.
484-
485-
### Real example: StarDist 3-D, with model caching
486-
487-
Heavy models must be loaded **once**, not per tile. On SLURM each tile is its
488-
own process so this matters less, but for local runs one process segments many
489-
tiles β€” cache the model at module level (or with `functools.lru_cache`):
490-
491-
```python
492-
# stardist_seg.py
493-
import numpy as np
494-
495-
_MODEL = None
496-
497-
def _model():
498-
global _MODEL
499-
if _MODEL is None: # loaded once per worker process
500-
from stardist.models import StarDist3D
501-
_MODEL = StarDist3D.from_pretrained("3D_demo")
502-
return _MODEL
503-
504-
def segment(tile: np.ndarray, prob_thresh: float = 0.5) -> np.ndarray:
505-
from csbdeep.utils import normalize
506-
507-
labels, _ = _model().predict_instances(
508-
normalize(tile), prob_thresh=prob_thresh
509-
)
510-
return labels.astype("int32")
511-
```
512-
513-
```yaml
514-
method: "custom"
515-
label_name: "stardist"
516-
custom:
517-
module: "stardist_seg"
518-
function: "segment"
519-
kwargs:
520-
prob_thresh: 0.5
521-
```
522-
523-
Using a GPU? The segment jobs already hold one (the `gres: "gpu:1"` request),
524-
so just let your framework see it β€” nothing extra in the config.
525-
526-
### Test it before you submit
527-
528-
Run your function on one real tile first β€” it catches shape/dtype bugs in
529-
seconds instead of after a queue wait. Output must be integer, same shape, `0`
530-
for background:
531-
532-
```python
533-
from patchworks import load_ome_zarr
534-
from my_seg import segment
535-
536-
img = load_ome_zarr("results/image.zarr", channel=0, level=0)
537-
tile = img[:, :512, :512].compute() # a small spatial block
538-
out = segment(tile)
539-
540-
assert out.shape == tile.shape, (out.shape, tile.shape)
541-
assert out.dtype.kind in "iu" # integer labels, not a float mask
542-
print("objects in tile:", int(out.max()))
543-
```
335+
Not using Cellpose? See [Custom segmentation function](custom_segmentation.md)
336+
for the function contract, examples (including label dilation), and how to
337+
test it before submitting. The rest of this section covers what's specific to
338+
running it **on the cluster**: getting the module importable and the
339+
checklist below.
544340

545341
### Make it importable on the cluster
546342

β€Žsrc/patchworks/plugins/dog.pyβ€Ž

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
>>> result = tile_process("image.zarr", fn, tile_shape=(1, 2048, 2048), overlap=32)
2626
2727
For the Snakemake workflow's ``method: "custom"`` (see
28-
docs/guide/snakemake.md "Custom segmentation function"), use the
28+
docs/guide/custom_segmentation.md), use the
2929
:func:`segment` adapter instead of the factory directly:
3030
3131
>>> # custom: {module: "patchworks.plugins.dog", function: "segment",

0 commit comments

Comments
Β (0)