Merge branch 'main' into ruff-test

Author: zm711

README.md

@@ -134,4 +134,4 @@ If you find SpikeInterface useful in your research, please cite:
 ```
 
 Please also cite other relevant papers for the specific components you use.
-For a ful list of references, please check the [references](https://spikeinterface.readthedocs.io/en/latest/references.html) page.
+For a full list of references, please check the [references](https://spikeinterface.readthedocs.io/en/latest/references.html) page.

doc/how_to/handle_drift.rst

@@ -1455,38 +1455,32 @@ Plot the results
 We load back the results and use the widgets module to explore the
 estimated drift motion.
 
-For all methods we have 4 plots: \* top left: time vs estimated peak
-depth \* top right: time vs peak depth after motion correction \* bottom
-left: the average motion vector across depths and all motion across
-spatial depths (for non-rigid estimation) \* bottom right: if motion
-correction is non rigid, the motion vector across depths is plotted as a
-map, with the color code representing the motion in micrometers.
-
-A few comments on the figures: \* the preset **‘rigid_fast’** has only
-one motion vector for the entire probe because it is a “rigid” case. The
-motion amplitude is globally underestimated because it averages across
-depths. However, the corrected peaks are flatter than the non-corrected
-ones, so the job is partially done. The big jump at=600s when the probe
-start moving is recovered quite well. \* The preset **kilosort_like**
-gives better results because it is a non-rigid case. The motion vector
-is computed for different depths. The corrected peak locations are
-flatter than the rigid case. The motion vector map is still be a bit
-noisy at some depths (e.g around 1000um). \* The preset **dredge** is
-offcial DREDge re-implementation in spikeinterface. It give the best
-result : very fast and smooth motion estimation. Very few noise. This
-method also capture very well the non rigid motion gradient along the
-probe. The best method on the market at the moement. An enormous thanks
-to the dream team : Charlie Windolf, Julien Boussard, Erdem Varol, Liam
-Paninski. Note that in the first part of the recording before the
-imposed motion (0-600s) we clearly have a non-rigid motion: the upper
-part of the probe (2000-3000um) experience some drifts, but the lower
-part (0-1000um) is relatively stable. The method defined by this preset
-is able to capture this. \* The preset **nonrigid_accurate** this is the
-ancestor of “dredge” before it was published. It seems to give the good
-results on this recording but with bit more noise. \* The preset
-**dredge_fast** similar than dredge but faster (using grid_convolution).
-\* The preset **nonrigid_fast_and_accurate** a variant of
-nonrigid_accurate but faster (using grid_convolution).
+For all methods we have 4 plots:
+    * top left: time vs estimated peak
+    * top right: time vs peak depth after motion correction
+    * bottom left: the average motion vector across depths and all motion across spatial depths (for non-rigid estimation)
+    * bottom right: if motion correction is non rigid, the motion vector across depths is plotted as a map,
+      with the color code representing the motion in micrometers.
+
+A few comments on the figures:
+    * the preset **‘rigid_fast’** has only one motion vector for the entire probe because it is a “rigid” case. The
+      motion amplitude is globally underestimated because it averages across depths. However, the corrected peaks
+      are flatter than the non-corrected ones, so the job is partially done. The big jump at=600s when the probe start
+      moving is recovered quite well.
+    * The preset **kilosort_like** gives better results because it is a non-rigid case. The motion vector is computed
+      for different depths. The corrected peak locations are flatter than the rigid case. The motion vector map is still
+      be a bit noisy at some depths (e.g around 1000um).
+    * The preset **dredge** is official DREDge re-implementation in spikeinterface. It give the best result : very fast
+      and smooth motion estimation. Very few noise. This method also capture very well the non rigid motion gradient along
+      the probe. The best method on the market at the moement. An enormous thanks to the dream team : Charlie Windolf,
+      Julien Boussard, Erdem Varol, Liam Paninski. Note that in the first part of the recording before the imposed motion
+      (0-600s) we clearly have a non-rigid motion: the upper part of the probe (2000-3000um) experience some drifts, but the
+      lower part (0-1000um) is relatively stable. The method defined by this preset is able to capture this.
+    * The preset **nonrigid_accurate** this is the ancestor of “dredge” before it was published. It seems to give good results
+      on this recording but with bit more noise.
+    * The preset **dredge_fast** similar than dredge but faster (using grid_convolution).
+    * The preset **nonrigid_fast_and_accurate** a variant of
+      nonrigid_accurate but faster (using grid_convolution).
 
 .. code:: ipython3
 

doc/how_to/process_by_channel_group.rst

@@ -97,12 +97,13 @@ to any preprocessing function.
     shifted_recordings = spre.phase_shift(split_recording_dict)
     filtered_recording = spre.bandpass_filter(shifted_recording)
     referenced_recording = spre.common_reference(filtered_recording)
+    good_channels_recording = spre.detect_and_remove_bad_channels(filtered_recording)
 
 We can then aggregate the recordings back together using the ``aggregate_channels`` function
 
 .. code-block:: python
 
-    combined_preprocessed_recording = aggregate_channels(referenced_recording)
+    combined_preprocessed_recording = aggregate_channels(good_channels_recording)
 
 Now, when ``combined_preprocessed_recording`` is used in sorting, plotting, or whenever
 calling its :py:func:`~get_traces` method, the data will have been

doc/modules/preprocessing.rst

@@ -247,6 +247,18 @@ interpolated with the :code:`interpolate_bad_channels()` function (channels labe
     # Case 2 : interpolate then
     rec_clean = interpolate_bad_channels(recording=rec, bad_channel_ids=bad_channel_ids)
 
+Once you have tested these functions and decided on your workflow, you can use the `detect_and_*`
+functions to do everything at once. These return a Preprocessor class, so are consistent with
+the "chain" concept for this module. For example:
+
+.. code-block:: python
+
+    # detect and remove bad channels
+    rec_only_good_channels = detect_and_remove_bad_channels(recording=rec)
+
+    # detect and interpolate the bad channels
+    rec_interpolated_channels = detect_and_interpolate_bad_channels(recording=rec)
+
 
 * :py:func:`~spikeinterface.preprocessing.detect_bad_channels()`
 * :py:func:`~spikeinterface.preprocessing.interpolate_bad_channels()`

examples/how_to/handle_drift.py

@@ -133,32 +133,33 @@ def preprocess_chain(rec):
 # We load back the results and use the widgets module to explore the estimated drift motion.
 #
 # For all methods we have 4 plots:
+#
 #   * top left: time vs estimated peak depth
 #   * top right: time vs peak depth after motion correction
 #   * bottom left: the average motion vector across depths and all motion across spatial depths (for non-rigid estimation)
 #   * bottom right: if motion correction is non rigid, the motion vector across depths is plotted as a map, with the color code representing the motion in micrometers.
 #
 # A few comments on the figures:
-# * the preset **'rigid_fast'** has only one motion vector for the entire probe because it is a "rigid" case.
-#   The motion amplitude is globally underestimated because it averages across depths.
-#   However, the corrected peaks are flatter than the non-corrected ones, so the job is partially done.
-#   The big jump at=600s when the probe start moving is recovered quite well.
-# * The preset **kilosort_like** gives better results because it is a non-rigid case.
-#   The motion vector is computed for different depths.
-#   The corrected peak locations are flatter than the rigid case.
-#   The motion vector map is still be a bit noisy at some depths (e.g around 1000um).
-# * The preset **dredge** is offcial DREDge re-implementation in spikeinterface.
-#   It give the best result : very fast and smooth motion estimation. Very few noise.
-#   This method also capture very well the non rigid motion gradient along the probe.
-#   The best method on the market at the moement.
-#   An enormous thanks to the dream team :  Charlie Windolf, Julien Boussard, Erdem Varol, Liam Paninski.
-#   Note that in the first part of the recording before the imposed motion (0-600s) we clearly have a non-rigid motion:
-#   the upper part of the probe (2000-3000um) experience some drifts, but the lower part (0-1000um) is relatively stable.
-#   The method defined by this preset is able to capture this.
-# * The preset **nonrigid_accurate** this is the ancestor of "dredge" before it was published.
-#   It seems to give the good results on this recording but with bit more noise.
-# * The preset **dredge_fast** similar than dredge but faster (using grid_convolution).
-# * The preset **nonrigid_fast_and_accurate** a variant of nonrigid_accurate but faster (using grid_convolution).
+#   * the preset **'rigid_fast'** has only one motion vector for the entire probe because it is a "rigid" case.
+#     The motion amplitude is globally underestimated because it averages across depths.
+#     However, the corrected peaks are flatter than the non-corrected ones, so the job is partially done.
+#     The big jump at=600s when the probe start moving is recovered quite well.
+#   * The preset **kilosort_like** gives better results because it is a non-rigid case.
+#     The motion vector is computed for different depths.
+#     The corrected peak locations are flatter than the rigid case.
+#     The motion vector map is still be a bit noisy at some depths (e.g around 1000um).
+#   * The preset **dredge** is offcial DREDge re-implementation in spikeinterface.
+#     It give the best result : very fast and smooth motion estimation. Very few noise.
+#     This method also capture very well the non rigid motion gradient along the probe.
+#     The best method on the market at the moement.
+#     An enormous thanks to the dream team :  Charlie Windolf, Julien Boussard, Erdem Varol, Liam Paninski.
+#     Note that in the first part of the recording before the imposed motion (0-600s) we clearly have a non-rigid motion:
+#     the upper part of the probe (2000-3000um) experience some drifts, but the lower part (0-1000um) is relatively stable.
+#     The method defined by this preset is able to capture this.
+#   * The preset **nonrigid_accurate** this is the ancestor of "dredge" before it was published.
+#     It seems to give the good results on this recording but with bit more noise.
+#   * The preset **dredge_fast** similar than dredge but faster (using grid_convolution).
+#   * The preset **nonrigid_fast_and_accurate** a variant of nonrigid_accurate but faster (using grid_convolution).
 #
 #
 
@@ -205,6 +206,7 @@ def preprocess_chain(rec):
 # We can see here that some clusters seem to be more compact on the 'y' axis, especially for the preset "nonrigid_accurate".
 #
 # Be aware that there are two ways to correct for the motion:
+#
 #   1. Interpolate traces and detect/localize peaks again  (`interpolate_recording()`)
 #   2. Compensate for drifts directly on peak locations (`correct_motion_on_peaks()`)
 #

src/spikeinterface/benchmark/benchmark_matching.py

@@ -38,9 +38,11 @@ def run(self, **job_kwargs):
         self.result = {"sorting": sorting, "spikes": spikes}
         self.result["templates"] = self.templates
 
-    def compute_result(self, with_collision=False, **result_params):
+    def compute_result(self, with_collision=False, match_score=0.5, exhaustive_gt=True):
         sorting = self.result["sorting"]
-        comp = compare_sorter_to_ground_truth(self.gt_sorting, sorting, exhaustive_gt=True)
+        comp = compare_sorter_to_ground_truth(
+            self.gt_sorting, sorting, exhaustive_gt=exhaustive_gt, match_score=match_score
+        )
         self.result["gt_comparison"] = comp
         if with_collision:
             self.result["gt_collision"] = CollisionGTComparison(self.gt_sorting, sorting, exhaustive_gt=True)

src/spikeinterface/benchmark/benchmark_merging.py

@@ -35,9 +35,11 @@ def run(self, **job_kwargs):
 
         self.result["sorting"] = merged_analyzer.sorting
 
-    def compute_result(self, **result_params):
+    def compute_result(self, match_score=0.5, exhaustive_gt=True):
         sorting = self.result["sorting"]
-        comp = compare_sorter_to_ground_truth(self.gt_sorting, sorting, exhaustive_gt=True)
+        comp = compare_sorter_to_ground_truth(
+            self.gt_sorting, sorting, exhaustive_gt=exhaustive_gt, match_score=match_score
+        )
         self.result["gt_comparison"] = comp
 
     _run_key_saved = [("sorting", "sorting"), ("merges", "pickle"), ("merged_pairs", "pickle"), ("outs", "pickle")]

src/spikeinterface/benchmark/benchmark_sorter.py

@@ -26,10 +26,12 @@ def run(self):
         sorting = NumpySorting.from_sorting(raw_sorting)
         self.result = {"sorting": sorting}
 
-    def compute_result(self, exhaustive_gt=True):
+    def compute_result(self, match_score=0.5, exhaustive_gt=True):
         # run becnhmark result
         sorting = self.result["sorting"]
-        comp = compare_sorter_to_ground_truth(self.gt_sorting, sorting, exhaustive_gt=exhaustive_gt)
+        comp = compare_sorter_to_ground_truth(
+            self.gt_sorting, sorting, exhaustive_gt=exhaustive_gt, match_score=match_score
+        )
         self.result["gt_comparison"] = comp
 
     _run_key_saved = [

src/spikeinterface/core/analyzer_extension_core.py

@@ -9,8 +9,6 @@
   * ComputeNoiseLevels which is very convenient to have
 """
 
-import warnings
-
 import numpy as np
 
 from .sortinganalyzer import AnalyzerExtension, register_result_extension

src/spikeinterface/core/baserecording.py

@@ -4,7 +4,7 @@
 from pathlib import Path
 
 import numpy as np
-from probeinterface import Probe, ProbeGroup, read_probeinterface, select_axes, write_probeinterface
+from probeinterface import read_probeinterface, write_probeinterface
 
 from .base import BaseSegment
 from .baserecordingsnippets import BaseRecordingSnippets