Skip to content

Commit 2e95d75

Browse files
committed
merge dev into main
Splits the cluster-workflow guide into focused pages (custom segmentation, measurements, label relations) and trims snakemake.md down to SLURM/cluster-specific content.
2 parents 0ad1072 + 3aa969d commit 2e95d75

8 files changed

Lines changed: 274 additions & 229 deletions

File tree

docs/examples/dog.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -97,12 +97,12 @@ tile_process(IMAGE, fn, tile_shape=(1, 1024, 1024), overlap=8, write_to=OUTPUT)
9797

9898
On the cluster, set `dilate: 2` in the YAML config instead — it applies to
9999
`method: "custom"` (this plugin) the same way it does for `cellpose`/
100-
`threshold`, see [Growing labels after segmentation](../guide/snakemake.md#growing-labels-afterwards-dilation).
100+
`threshold`, see [Growing labels afterwards](../guide/custom_segmentation.md#growing-labels-afterwards-dilation).
101101

102102
## Using it in the Snakemake workflow
103103

104104
No dedicated wiring needed — `patchworks.plugins.dog` exposes a `segment(tile, **kwargs)`
105-
adapter for the documented [`"custom"` method](../guide/snakemake.md#custom-segmentation-function):
105+
adapter for the documented [`"custom"` method](../guide/custom_segmentation.md):
106106

107107
```yaml
108108
method: "custom"
@@ -185,6 +185,6 @@ Checklist specific to this config:
185185

186186
Segment the cell body with Cellpose and the cilia with `dog_label_fn` as two
187187
separate `tile_process` runs (same image, same `tile_shape`), then use
188-
[`label_relations`](../guide/snakemake.md#relating-labels-across-segmentations)
188+
[`label_relations`](../guide/label_relations.md)
189189
to map each cilium to the cell it belongs to — see
190190
`workflow/config/multi.yaml` for the same thing wired up as a cluster job.

docs/guide/custom_segmentation.md

Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Custom segmentation function
2+
3+
Not using Cellpose? Run **your own** per-tile function — no need to edit the
4+
package. You write one function; patchworks handles everything around it
5+
(tiling, halos, skipping empty tiles, the zarr-native merge, global
6+
relabelling, resume, logs).
7+
8+
This applies the same whether you call the API directly (`tile_process`) or
9+
run via the [Snakemake cluster workflow](snakemake.md) — see [Wiring it into
10+
the cluster workflow](#wiring-it-into-the-cluster-workflow) below for the
11+
config side.
12+
13+
## The contract
14+
15+
Your function is called **once per tile**:
16+
17+
```python
18+
labels = segment(tile) # plus any kwargs you configure
19+
```
20+
21+
| | What you get / must return |
22+
| --- | --- |
23+
| **Input `tile`** | A NumPy array of **one** tile, with the overlap halo already included. The channel and pyramid level are already selected, so it is purely spatial: `(z, y, x)` for a 3-D run, `(y, x)` for 2-D. Dtype is the image's (e.g. `uint16`). |
24+
| **Return** | An integer **label** array (not a boolean mask), **same shape** as `tile`. `0` = background; each object a distinct positive integer. |
25+
| **Labels** | Only need to be unique **within the tile**. Don't try to make them globally unique — the merge step stitches objects across tile borders and renumbers everything to a contiguous `1..N` (`sequential_labels: true`). |
26+
| **Shape** | Must match the input exactly — patchworks trims the halo off your output, so a wrong shape is an error. Don't crop or resize inside the function. |
27+
28+
That is the whole interface. Anything that turns an image tile into a label
29+
image works: classic image processing, StarDist, a trained model, an external
30+
binary you shell out to, …
31+
32+
## Minimal example (no GPU, no deps beyond scikit-image)
33+
34+
```python
35+
# my_seg.py
36+
import numpy as np
37+
from skimage.measure import label
38+
39+
def segment(tile: np.ndarray, sigma: float = 2.0) -> np.ndarray:
40+
"""Threshold + connected components. Returns int32 labels (0 = bg)."""
41+
from skimage.filters import gaussian, threshold_otsu
42+
43+
smooth = gaussian(tile, sigma=sigma, preserve_range=True)
44+
thr = threshold_otsu(smooth) if smooth.max() > smooth.min() else np.inf
45+
return label(smooth > thr).astype("int32")
46+
```
47+
48+
```python
49+
from patchworks import tile_process
50+
from my_seg import segment
51+
52+
tile_process("image.zarr", segment, write_to="labels.zarr")
53+
```
54+
55+
## Growing labels afterwards (dilation)
56+
57+
To grow every label by a few pixels after segmentation, wrap your function
58+
with [`patchworks.dilate_labels`](../api/postprocess.md):
59+
60+
```python
61+
from patchworks import tile_process, dilate_labels
62+
from patchworks.plugins.dog import dog_label_fn
63+
64+
fn = dog_label_fn(low_sigma=1.0, high_sigma=3.0, threshold=0.02)
65+
fn = dilate_labels(fn, iterations=2) # grow each label by 2 px, then run
66+
result = tile_process("image.zarr", fn, tile_shape=(1, 2048, 2048),
67+
overlap=8, write_to="labels.zarr")
68+
```
69+
70+
`dilate_labels` wraps any `(tile) -> labels` function — the same contract
71+
above — so it works with `dog_label_fn`, `cellpose_fn`, or your own
72+
`segment`. It dilates each tile's labels before the halo is trimmed and
73+
tiles are merged, so `overlap` must still cover the dilation amount. On the
74+
cluster, set `dilate: N` in the config instead — see [Configure the
75+
run](snakemake.md#3-configure-the-run).
76+
77+
## Real example: StarDist 3-D, with model caching
78+
79+
Heavy models must be loaded **once**, not per tile. On SLURM each tile is its
80+
own process so this matters less, but for local runs one process segments many
81+
tiles — cache the model at module level (or with `functools.lru_cache`):
82+
83+
```python
84+
# stardist_seg.py
85+
import numpy as np
86+
87+
_MODEL = None
88+
89+
def _model():
90+
global _MODEL
91+
if _MODEL is None: # loaded once per worker process
92+
from stardist.models import StarDist3D
93+
_MODEL = StarDist3D.from_pretrained("3D_demo")
94+
return _MODEL
95+
96+
def segment(tile: np.ndarray, prob_thresh: float = 0.5) -> np.ndarray:
97+
from csbdeep.utils import normalize
98+
99+
labels, _ = _model().predict_instances(
100+
normalize(tile), prob_thresh=prob_thresh
101+
)
102+
return labels.astype("int32")
103+
```
104+
105+
Using a GPU? Just let your framework see it — nothing extra needed.
106+
107+
## Test it before you submit
108+
109+
Run your function on one real tile first — it catches shape/dtype bugs in
110+
seconds instead of after a queue wait (or a long local run). Output must be
111+
integer, same shape, `0` for background:
112+
113+
```python
114+
from patchworks import load_ome_zarr
115+
from my_seg import segment
116+
117+
img = load_ome_zarr("results/image.zarr", channel=0, level=0)
118+
tile = img[:, :512, :512].compute() # a small spatial block
119+
out = segment(tile)
120+
121+
assert out.shape == tile.shape, (out.shape, tile.shape)
122+
assert out.dtype.kind in "iu" # integer labels, not a float mask
123+
print("objects in tile:", int(out.max()))
124+
```
125+
126+
## Wiring it into the cluster workflow
127+
128+
Point the config at your module and function:
129+
130+
```yaml
131+
method: "custom"
132+
label_name: "my_labels"
133+
custom:
134+
module: "my_seg" # import name
135+
function: "segment" # default is "segment"
136+
kwargs: # optional — forwarded as segment(tile, **kwargs)
137+
sigma: 1.5
138+
```
139+
140+
Same for the StarDist example above:
141+
142+
```yaml
143+
method: "custom"
144+
label_name: "stardist"
145+
custom:
146+
module: "stardist_seg"
147+
function: "segment"
148+
kwargs:
149+
prob_thresh: 0.5
150+
```
151+
152+
For getting the module importable on a compute node, dependency/GPU
153+
checklist, and troubleshooting, see [Custom functions on the
154+
cluster](snakemake.md#custom-segmentation-function) in the cluster guide.

docs/guide/label_relations.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# Relating labels across segmentations
2+
3+
Once you have two segmentations of the same image (e.g. nuclei inside cells),
4+
`label_relations()` maps each label in one to the label it overlaps most in
5+
the other — by streaming both arrays chunk by chunk, so it scales to
6+
hundreds of thousands of objects without loading anything fully into RAM.
7+
8+
Both label arrays must share the exact same chunk layout — same
9+
`tile_shape`/pyramid `level` when they were produced.
10+
11+
```python
12+
import dask.array as da
13+
from patchworks import label_relations
14+
15+
nuclei = da.from_zarr("results/image.zarr", component="labels/nuclei_labels/0")
16+
cells = da.from_zarr("results/image.zarr", component="labels/cyto_labels/0")
17+
18+
table = label_relations(nuclei, cells)
19+
table[2]
20+
# {'match': 3, 'overlap_voxels': 4821, 'overlap_fraction': 0.94}
21+
# -> nucleus 2 belongs to cell 3, 94% of its voxels fall inside it
22+
```
23+
24+
`table` only contains matched `a` labels (nuclei with at least one
25+
overlapping voxel in `cells`) — unmatched labels and full per-`b` coverage
26+
need a bit more bookkeeping (the [cluster workflow's `run_multi.py`
27+
script](snakemake.md#one-command-multiple-segmentations--relations) does
28+
this for you and writes it as a two-sheet workbook).
29+
30+
Save it as a table yourself:
31+
32+
```python
33+
import csv
34+
35+
with open("nuclei_to_cell.csv", "w", newline="") as f:
36+
w = csv.writer(f)
37+
w.writerow(["nucleus_id", "cell_id", "overlap_voxels", "overlap_fraction"])
38+
for nucleus_id, m in table.items():
39+
w.writerow([nucleus_id, m["match"], m["overlap_voxels"], m["overlap_fraction"]])
40+
```
41+
42+
On the cluster, producing the two label stores in the first place is a
43+
matter of running the workflow twice against the same `work_dir` — see
44+
[Running two segmentations](snakemake.md#running-two-segmentations-eg-nuclei--cytoplasm).

docs/guide/measurements.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# Measurements (fast, whole-volume regionprops)
2+
3+
`skimage.measure.regionprops` needs the full labelled + intensity array in
4+
RAM — fine for one tile, not for a hundred-thousand-object OME-ZARR.
5+
6+
## Interactively, in napari
7+
8+
[napari-chunked-regionprops](https://github.com/imcf/napari-chunked-regionprops)
9+
is built for this — its "Measure" dock widget computes area/centroid/intensity
10+
stats directly off a Labels layer's dask/zarr-backed array, out-of-core, and
11+
scales with chunk count rather than object count. It's the best fit for
12+
measuring *every* object in a store this size, not just a cropped region —
13+
see [View image + labels in napari](ome_zarr_napari.md#view-image--labels-in-napari).
14+
Bundled in `patchworks[napari]`.
15+
16+
For interactively inspecting individual cells by clicking in the viewer (not
17+
all objects at once), the
18+
[napari-skimage-regionprops](https://github.com/haesleinhuepf/napari-skimage-regionprops)
19+
plugin's table widget also works well — point it at a cropped region rather
20+
than the full volume, since it loads its input fully into memory.
21+
22+
## Headless / scripted
23+
24+
Use [`dask-image`](https://image.dask.org)'s `ndmeasure`, which computes
25+
directly on the dask/zarr-backed arrays, chunk-parallel, without
26+
materializing the volume:
27+
28+
```bash
29+
pip install dask-image
30+
```
31+
32+
```python
33+
import dask.array as da
34+
from dask_image.ndmeasure import area, center_of_mass, mean, standard_deviation
35+
36+
labels = da.from_zarr("results/image.zarr", component="labels/cyto_labels/0")
37+
image = da.from_zarr("results/image.zarr", component="0")[0] # channel 0, level 0
38+
39+
ids = da.unique(labels[labels > 0]).compute()
40+
areas = area(image, labels, ids).compute() # voxel counts
41+
means = mean(image, labels, ids).compute() # mean intensity
42+
stds = standard_deviation(image, labels, ids).compute()
43+
centroids = center_of_mass(image, labels, ids).compute() # voxel coords (z, y, x)
44+
```
45+
46+
Multiply `areas` by the voxel's physical volume and `centroids` by the pixel
47+
size (both read straight from the OME-ZARR's own `multiscales` metadata) to
48+
get µm-scale measurements.

docs/guide/ome_zarr_napari.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ just one channel instead: `view_in_napari("scan.zarr", channel=0)`.
159159
out-of-core straight off the Labels layer's backing dask/zarr array, so
160160
it scales to the same huge label images `tile_process` writes, unlike
161161
plain `skimage.measure.regionprops`. Bundled in `patchworks[napari]`. See
162-
[Measurements](snakemake.md#measurements-fast-whole-volume-regionprops)
162+
[Measurements](measurements.md)
163163
for the non-interactive/headless equivalent.
164164

165165
## End-to-end

0 commit comments

Comments
 (0)