snt: update integration w/ machine learning tools

update curation

Author: tferr

_pages/plugins/snt/comp-tools.md

@@ -8,6 +8,10 @@ forum-tag: snt
 update-site: Neuroanatomy
 ---
 
+### AI Tools
+Described in [Seeded Tracing](./seeds).
+
+
 ### ImageJ Ecosystem
 
 | Tool | Description | Comments |

_pages/plugins/snt/curation.md

@@ -171,6 +171,24 @@ The table displays warnings sorted by severity (Errors first, then Warnings, the
 **Right-click** the table for additional options: clearing all warnings, setting the visiting zoom level, resetting zoom to the current image magnification, and detaching/docking the table into a separate floating window.
 
 
+## Seed Review
+The Curation Assistant also accepts warnings contributed by external programs: Warnings produced outside SNT can be imported as [Warning seeds](./seeds), and loaded into the assistant. These entries:
+
+- Carry the check name "Seed Review" so they're easier to spot (and filter) among the algorithmically-generated warnings
+- Use the seed's 3D position as the warning location, so the standard double-click-to-navigate behavior flies the canvas to the seed.
+- Their severity level (Info / Warning / Error) is manually chosen: Use whichever level matches the review intent
+- Encode the seed's confidence, type, and source in the message
+
+A typical workflow:
+
+1. Import seeds into the [Seeded Tracing Assistant](./seeds#importing-seeds) (e.g., a CSV file of predicted problematic locations generated by a deep-learning pipeline)
+2. Adjust the [confidence range](./seeds#confidence-filtering) and sort the table so the candidates of interest are at the top
+3. Select the rows to be reviewed, right-click, and choose _Send Selected Seeds to Curation Assistant..._ → _As Warning_ (or _As Info Message_ / _As Error_, as appropriate)
+4. The action will import the selected seeds and activate to the Assistant tab so that you can step through the new entries via double-click navigation
+
+**NB**: Imported seeds are not de-duplicated: sending the same selection twice produces two rows
+
+
 # Detecting Crossovers
 
 A crossover is a spot where at least two neurites pass very close to each other in space (so they may look like they intersect in the image) but they are not connected in the reconstructed graph (i.e., there is no shared node / true topological join). Identification of crossover sites is thus useful to disambiguate overlaps between neurites and spot possible tracing mistakes, such as missed branch-points or false merges.

_pages/plugins/snt/faq.md

@@ -80,6 +80,9 @@ Yes. You can trace using the the XY,ZY,XZ [views](/plugins/snt/walkthroughs#accu
 ### Can I undo tracing mistakes?
 Yes. With version 5 and later, temporary segments can be reverted before finishing a path simply by pressing {% include key key='Z' %}. In Edit Mode, editing operations can also be reverted using the same shortcut. By default, the last 20 operations can be undone.
 
+### Does SNT use AI?
+See [machine learning](./machine-learning).
+
 ### Having to confirm individual segments is too cumbersome. Is it possible to trace without interruption, by clicking in succession?
 Yes. Uncheck the *Confirm temporary segments* in the *Options* tab (*Temporary Paths* section): When this option is disabled. Segments are confirmed automatically. Note that double-click finishes a path directly, and the last confirmed segment can always be undone with the {% include key key='Z' %} key.
 
@@ -136,3 +139,6 @@ See [Spine/Varicosity/Label Analysis](./spines-varicosities). In-depth quantific
 
 ### Can SNT reconstruct somata?
 Yes, see [Autotracing › Soma/Root Detection](./auto-tracing#somaroot-detection). Note, however, that most SNT tutorials favor the [single-point soma representation](https://neuromorpho.org/SomaFormat.html).
+
+### Can I use cellpose segmentations?
+Yes, see [Machine learning](./machine-learning).

_pages/plugins/snt/machine-learning.md

@@ -12,16 +12,95 @@ doi:
 ---
 
 {% capture version%}
-**This page was last revised for [version 5.0.0](https://github.com/morphonets/SNT/releases)**.
+**This page was last revised for [version 5.1.0](https://github.com/morphonets/SNT/releases)**.
 {% endcapture %}
 {% include notice content=version %}
 
-SNT interacts with [Labkit](../labkit) and [Trainable Weka Segmentation (TWS)](../tws), both leveraging  machine learning algorithms for semantic segmentation of images (namely, random forest classifiers). The bridge between these tools makes it possible to:
+SNT integrates with deep-learning (DL) segmentation tools (e.g., [cellpose](https://www.cellpose.org), [CSBDeep](/plugins/csbdeep), [DeepImageJ](https://deepimagej.github.io/), [nnU-Net](https://github.com/MIC-DKFZ/nnUNet), [StarDist](https://stardist.net/), and others) in both directions: their outputs — segmentations, probability maps, detected coordinates — become inputs to SNT's tracing and analysis workflows, while SNT's reconstructions, after [curation](./curation) and [refinement](./manual#refinefit-), can be exported as ground-truth labels to train the next round of models. For simpler, classical-ML alternatives, SNT also embeds ensemble (random-forest) classifiers via [Labkit/TWS](#embedded-tools-labkit--tws). The sections below provide an overview of how model predictions can be consumed in SNT, and how SNT can be used to generate training data for either family of models.
+
+# Consuming DL Predictions
+
+## Probability Maps
+
+A segmentation network's per-voxel probability output can be loaded as a [secondary tracing layer](./manual#tracing-on-secondary-image-layer) alongside the raw image.
+SNT's [autotracers](./auto-tracing) can then use the probability map either as the primary signal (tracing directly on the network's prediction) or as a **score map** that guides pruning and tip extension during autotracing (see [Branch Filtering and Scoring](./auto-tracing#branch-filtering-and-scoring)).
+With semi-automated tracing, the secondary layer is opened using the [Interactive Tracing](./manual#interactive-tracing) widget in the Main tab. With [auto-tracing](./auto-tracing), the score map is loaded/chosen as a [parameter](./auto-tracing#branch-filtering-and-scoring).
+
+Note that any tool that can save a probability map ([Labkit/TWS](#embedded-tools-labkit--tws), custom PyTorch/TensorFlow scripts) is supported: SNT only sees the resulting image.
+
+## Seed Points
+
+Many DL detectors output masks, or _candidate locations_: lists of soma centres, axon-terminal candidates, branch points, etc.
+SNT consumes these through the [Seeded Tracing Assistant](./seeds#seeded-tracing-assistant), where each candidate location/masked object lives as a 3D point with an associated confidence, optional radius, and free-form "type" and "source" fields.
+Once loaded, seeds can be:
+
+- [Filtered](./seeds#confidence-filtering) and curated interactively
+- Used to root automated traces: E.g., cellpose masks can be used to specify the soma locations when autotracing [multiple cells](./auto-tracing#multiple-cells)
+- Used as endpoint targets for a single-cell trace
+- Used as soft attractors that bias the routing of an automated trace through the predicted locations
+
+See [Seeded Tracing](./seeds) for full workflows. Note that the detector itself can be anything that produces a CSV list, or a labels image (from which centroids are extracted automatically).
+
+## Quality Control Locations
+
+The [Curation Assistant](./curation) accepts DL-flagged "suspect" locations (e.g., from a model trained on previously-reviewed reconstructions) as warnings: each predicted location is shown in the Assistant's [warnings table](./curation#warnings-table) for human review, mirroring how the Assistant's own [live monitors](./curation#live-monitoring-parameters) report issues from rule-based checks.
+
+Programmatic entry is via `CurationManager#addWarnings(List<Warning>)`. Each `Warning` is a record carrying a short check name, a severity (`INFO`, `WARNING`, `ERROR`), a human-readable message, an optional spatial location, the list of affected paths, and the measured value and threshold that triggered the entry. Once injected, predictions live alongside the Assistant's own findings: they can be [filtered by severity](./curation#filter-by-severity), navigated to on the canvas, tagged as accept / reject / unsure for [downstream training](./curation#seed-review), and visited in batch from the warnings table.
+
+```groovy
+import sc.fiji.snt.analysis.curation.PlausibilityCheck.Warning
+import sc.fiji.snt.analysis.curation.PlausibilityCheck.Severity
+import sc.fiji.snt.util.PointInImage
+
+// `predictions` is whatever a detector emits per suspect location
+def warnings = predictions.collect { p ->
+    new Warning("DL-QC", Severity.WARNING,
+            "Predicted error (score=${p.score})",
+            new PointInImage(p.x, p.y, p.z), // location
+            p.nearbyPaths,         // List<Path> list of affected paths
+            p.score, 0.5)          // measured value, threshold
+}
+ui.getCurationManager().addWarnings(warnings)
+```
+
+This turns the Curation Assistant into a generic target for any QC model that returns spatial predictions on a finished reconstruction, including models trained on the [Seed Review](./curation#seed-review) export produced by an earlier curation pass, closing the feedback loop between review and retraining.
+
+
+# Generating Training Data
+
+SNT's reconstruction, segmentation, and annotation tooling is also useful for _producing_ training data for downstream models. The data SNT generates is well-suited as ground truth: it's stored in calibrated coordinates, records provenance metadata, supports [post-hoc refinements](./manual#refinefit-), and can be reviewed in the [Curation Assistant](./curation) before being used to train downstream models.
+
+These aspects are covered elsewhere in the documentation:
+
+- **Pixel-level labels for segmentation networks**: Path Manager's {% include bc path='Process' %} menu exports selected paths as classifier labels into [Labkit/TWS](#embedded-tools-labkit--tws). See [Training Models](#training-models) below for the Weka-based workflow; the same exported labels can be repurposed for any U-Net-style trainer that consumes binary or multiclass images
+
+- **Path fitting for automated refinement**: The [Refine/Fit](./manual#refinefit-) command snaps each node onto the signal centerline and recomputes per-node radii from cross-sectional intensity profiles. The result is a geometry that hugs the actual neurite rather than the tracer's first guess, lifting label quality before export
+
+- **Editing tools for manual refinement**: SNT's [Edit Mode](./manual#edit-mode-8679-shifte) ({% include key keys='Shift|E' %}) supports per-node manipulation: insert, delete, redirect, or [paint](./manual#paint-mode-p) stretches of a path directly on the image. Auto-traced or imported paths can be corrected to match the underlying signal precisely, producing higher-fidelity ground truth than a raw automatic trace would offer
+
+- **Cross-over detections**: The [crossover detector](./curation#detecting-crossovers) flags places where neurites pass close together in 3D but aren't topologically joined. Reviewing and resolving these before export prevents ambiguities from leaking into the training datasets
+
+- **Curated reconstructions**: Paths reviewed in the [Curation Assistant](./curation) can be exported (with their accept/reject status preserved in path tags) as either positive examples for training models that learn correct topology, or negative examples for QC-classifier training. A scriptable bridge between the Assistant and a training pipeline facilitates this, e.g.,:
+
+```groovy
+import sc.fiji.snt.Tree
+import sc.fiji.snt.analysis.curation.CurationTags
+def tree = Tree.fromFile("/path/to/annotated/file.traces")
+def p = CurationTags.partitionByReviewStatus(tree)
+println "${p.positive().size()} +, ${p.negative().size()} -, ${p.unsure().size()} ?, ${p.unreviewed().size()}"
+```
+
+- **Coordinate-only labels for detectors**: the [Seeded Tracing Assistant](./seeds#seeded-tracing-assistant)'s export commands write CSV files that any soma- or endpoint-detection model can ingest as positive examples. Manual annotations, ROI-derived seeds, and label-image-derived seeds can all be stored in the same exportable format
+
+
+# Embedded Tools: Labkit & TWS
+
+SNT interacts with [Labkit](../labkit) and [Trainable Weka Segmentation (TWS)](../tws), both leveraging machine learning algorithms for semantic segmentation of images (namely, random forest classifiers). The bridge between these tools makes it possible to:
 
 - Import a pre-trained model into SNT and directly load the probability maps of the semantic segmentation as secondary tracing layer
 - Train a model with SNT paths
 
-The table below summarizes key differences between Labkit and TWS (as of SNT v4.3.0). Note that both tools classify images using the [Weka framework](https://ml.cms.waikato.ac.nz/weka/).
+The table below summarizes key differences between Labkit and TWS (as of SNT v5). Note that both tools classify images using the [Weka framework](https://ml.cms.waikato.ac.nz/weka/).
 
 |                                                      | **[Labkit](../labkit)**                              | **[TWS](../tws)**            |
 |------------------------------------------------------|------------------------------------------------------|------------------------------|
@@ -73,7 +152,7 @@ There are a couple of examples in SNT's neuroanatomy [template collection](scrip
 - *Apply Weka Model To Tracing Image*: Demonstrates how to apply a pre-existing Weka model to the image being traced
 - *Train Weka Classifier*:   Exemplifies how to train a Weka model using traced paths and ROIs 
 
-## References
+# References
 
 {% include citation %}