Stitching overlapping patch predictions

[ajinkya-kulkarni]

Hi, thanks for this repo. I was wondering if you already have or would be interested if I work/merge on stitching patch predictions into a big contiguous mask.

[lguerard]

Hi !

I'm not sure I understand exactly the request, patchworks already merge back all patch predictions into a big OME-ZARR object. Is that what you meant ? Otherwise, always happy to collaborate with people on this repo :)

[ajinkya-kulkarni]

I specifically mean the part of code in patchworks which handles multiple predicted nuclei (who can in principle differ amongst themselves slightly) in overlapping areas - how are they correctly resolved?

[lguerard]

Good question. To answer specifically about duplicate/differing nuclei in overlapping regions: patchworks actually sidesteps that problem rather than resolving it with IoU matching or similar.

Each tile is segmented with a small "halo", extra pixels borrowed from neighboring tiles, so the model has enough context to segment nuclei near tile edges properly. That halo is then trimmed off before the tiles are stitched together, so in the final result each pixel belongs to exactly one tile. There's no region where two tiles both produce a prediction for the same nucleus, so there's nothing to reconcile.

The only case left is a nucleus that sits exactly on a tile boundary and gets split into two different IDs (one per tile). That's handled by scanning the seam between tiles and merging any labels that directly touch across it via connected components.

So to be clear: this is touching-label merging only, not IoU-based overlap merging — that's an explicit design tradeoff, not something currently supported.

[ajinkya-kulkarni]

Thanks for the reply! Could you point me to the relevant code for this please?

[lguerard]

The halo/context part (adding overlap before segmentation, trimming it after) is in src/patchworks/_core.py (see the overlap parameter around line 135, and the trim step around line 484-489). The actual boundary-scan-and-merge step is _scan_touching_pairs in _merge.py, which checks for touching labels across chunk boundaries and merges them via connected components.