Skip to content

Commit f1e0ae9

Browse files
Implement primmitives for distributed graph computation (#58)
1 parent fdcf071 commit f1e0ae9

11 files changed

Lines changed: 1720 additions & 29 deletions

File tree

MIGRATION_GUIDE.md

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -490,6 +490,54 @@ Notes:
490490
endpoint ids.
491491
- Non-contiguous labels are copied to contiguous memory before entering C++.
492492

493+
### Distributed Region Adjacency Graphs and Features
494+
495+
For volumes too large to hold in memory, `nifty.distributed` builds a RAG and
496+
its edge features block by block and merges the results. `bioimage-cpp` provides
497+
the equivalent **low-level primitives** under `bic.graph.distributed`; the
498+
orchestration nifty bundles (iterating blocks, sizing halos, and serializing the
499+
per-block subgraphs/features to zarr/N5/HDF5) is intentionally left to the
500+
caller, since I/O and block scheduling belong in Python.
501+
502+
The primitives assume **globally consistent labels** (a segment has the same id
503+
in every block — as after a stitched distributed watershed). A block owns the
504+
pixel-pairs whose reference pixel lies in its inner (non-halo) box, so the
505+
caller reads each block with a halo (≥1 on the forward faces for the region
506+
graph / an edge map; ≥ `max |offset|` per side for affinities) and passes the
507+
owned box as `own_begin` / `own_shape` (e.g. from `bic.utils.Blocking`'s
508+
`get_block_with_halo(...).inner_block_local`).
509+
510+
```python
511+
d = bic.graph.distributed
512+
513+
# per block (labels read with a halo):
514+
edges = d.block_region_adjacency_edges(labels_block, own_begin, own_shape)
515+
block_edges, block_stats = d.block_edge_map_stats(labels_block, edge_map_block, own_begin, own_shape)
516+
block_edges, block_stats = d.block_affinity_stats(labels_block, aff_block, offsets, own_begin, own_shape)
517+
518+
# merge the graph, then build the global graph:
519+
global_edges = d.merge_edges([edges_block_0, edges_block_1, ...])
520+
graph = bic.graph.UndirectedGraph.from_unique_edges(number_of_nodes, global_edges)
521+
522+
# fold per-block features onto the global edges, then finalize:
523+
acc = d.empty_edge_stats(graph.number_of_edges)
524+
for be, bs in per_block_stats:
525+
acc = d.merge_block_edge_stats(graph, acc, be, bs)
526+
features = d.finalize_edge_features(acc, compute_complex_features=True)
527+
```
528+
529+
Notes:
530+
531+
- Blocked results reproduce the whole-volume `region_adjacency_graph` /
532+
`features.*_features` exactly for `size`, `min` and `max`, and to
533+
floating-point tolerance for `mean` / `std` (the running sums depend on thread
534+
count and merge order).
535+
- **Median and percentiles are not distributable.** The distributed complex
536+
output is the moment subset `[mean, std, min, max, size]` — the corresponding
537+
columns of the in-core 12-column complex features.
538+
- Making per-block-local ids globally consistent (label stitching) is a separate
539+
step and is not part of these primitives.
540+
493541
### Breadth-First Search
494542

495543
Nifty has an internal `BreadthFirstSearch` template used during lifted-edge

include/bioimage_cpp/detail/grid.hxx

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,4 +86,27 @@ inline bool is_valid_grid_edge(
8686
return valid_offset_target(node, offset, shape, strides, unused);
8787
}
8888

89+
// Given a signed offset `delta` along one axis and the axis `length`, return the
90+
// half-open range of reference coordinates `[lo, hi)` for which `coord + delta`
91+
// stays in `[0, length)`. Returns `lo >= hi` when the offset is larger than the
92+
// axis (no valid reference coordinate). This is the per-axis primitive behind
93+
// the offset-box sweeps in feature accumulation and the distributed block
94+
// extraction — it depends only on grid geometry, not on any array data.
95+
inline void valid_axis_range(
96+
const std::ptrdiff_t delta,
97+
const std::size_t length,
98+
std::size_t &lo,
99+
std::size_t &hi
100+
) {
101+
if (delta >= 0) {
102+
lo = 0;
103+
const auto d = static_cast<std::size_t>(delta);
104+
hi = (d >= length) ? 0 : (length - d);
105+
} else {
106+
const auto d = static_cast<std::size_t>(-delta);
107+
lo = (d >= length) ? length : d;
108+
hi = length;
109+
}
110+
}
111+
89112
} // namespace bioimage_cpp::detail
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#pragma once
2+
3+
#include <cstdint>
4+
#include <stdexcept>
5+
#include <type_traits>
6+
7+
namespace bioimage_cpp::detail {
8+
9+
// Convert a label value to a graph node id (`std::uint64_t`), rejecting negative
10+
// values for signed label dtypes. Shared by the region-adjacency-graph scan,
11+
// edge-feature accumulation, and the distributed block-extraction primitives so
12+
// they all treat labels identically.
13+
template <class T>
14+
std::uint64_t checked_label_to_node(const T value) {
15+
if constexpr (std::is_signed_v<T>) {
16+
if (value < 0) {
17+
throw std::invalid_argument("labels must not contain negative values");
18+
}
19+
}
20+
return static_cast<std::uint64_t>(value);
21+
}
22+
23+
} // namespace bioimage_cpp::detail

0 commit comments

Comments
 (0)