Merge branch 'main' into isosplit

Author: samuelgarcia

.pre-commit-config.yaml

@@ -1,6 +1,6 @@
 repos:
 -   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v5.0.0
+    rev: v6.0.0
     hooks:
     -   id: check-yaml
     -   id: end-of-file-fixer

doc/how_to/customize_a_plot.rst

@@ -1,3 +1,5 @@
+.. _customize-a-plot:
+
 Customize a plot
 ================
 

doc/how_to/handle_drift.rst

@@ -4,6 +4,9 @@
     %load_ext autoreload
     %autoreload 2
 
+
+.. _handle-drift-in-your-recording:
+
 Handle motion/drift in your recording
 =====================================
 

doc/how_to/physical_units.rst

@@ -1,3 +1,4 @@
+
 .. _physical_units:
 
 Work with physical units in SpikeInterface recordings

doc/modules/benchmark.rst

@@ -1,8 +1,7 @@
 Benchmark module
 ================
 
-This module contains machinery to compare some sorters against ground truth in many multiple situtation.
-
+This module contains machinery to compare some sorters against ground truth in multiple situations.
 
 ..notes::
 
@@ -13,22 +12,21 @@ This module contains machinery to compare some sorters against ground truth in m
 This module also aims to benchmark sorting components (detection, clustering, motion, template matching) using the
 same base class :py:func:`~spikeinterface.benchmark.BenchmarkStudy()` but specialized to a targeted component.
 
-By design, the main class handle the concept of "levels" : this allows to compare several complexities at the same time.
+By design, the main class handles the concept of "levels" : this allows you to compare several complexities at the same time.
 For instance, compare kilosort4 vs kilsort2.5 (level 0) for different noises amplitudes (level 1) combined with
-several motion vectors (leevel 2).
+several motion vectors (level 2).
 
 **Example: compare many sorters : a ground truth study**
 
 We have a high level class to compare many sorters against ground truth: :py:func:`~spikeinterface.benchmark.SorterStudy()`
 
-
-A study is a systematic performance comparison of several ground truth recordings with several sorters or several cases
+A study is a systematic performance comparison of several ground truth recordings with several sorters or several cases,
 like the different parameter sets.
 
 The study class proposes high-level tool functions to run many ground truth comparisons with many "cases"
 on many recordings and then collect and aggregate results in an easy way.
 
-The all mechanism is based on an intrinsic organization into a "study_folder" with several subfolders:
+The mechanism is based on an intrinsic organization into a "study_folder" with several subfolders:
 
   * datasets: contains ground truth datasets
   * sorters : contains outputs of sorters
@@ -39,24 +37,20 @@ The all mechanism is based on an intrinsic organization into a "study_folder" wi
 
 .. code-block:: python
 
-    import matplotlib.pyplot as plt
-    import seaborn as sns
-
-    import spikeinterface.extractors as se
+    import spikeinterface as si
     import spikeinterface.widgets as sw
     from spikeinterface.benchmark import SorterStudy
 
-
     # generate 2 simulated datasets (could be also mearec files)
-    rec0, gt_sorting0 = generate_ground_truth_recording(num_channels=4, durations=[30.], seed=42)
-    rec1, gt_sorting1 = generate_ground_truth_recording(num_channels=4, durations=[30.], seed=91)
+    rec0, gt_sorting0 = si.generate_ground_truth_recording(num_channels=4, durations=[30.], seed=42)
+    rec1, gt_sorting1 = si.generate_ground_truth_recording(num_channels=4, durations=[30.], seed=91)
 
     datasets = {
         "toy0": (rec0, gt_sorting0),
         "toy1": (rec1, gt_sorting1),
     }
 
-    # define some "cases" here we want to test tridesclous2 on 2 datasets and spykingcircus2 on one dataset
+    # define some "cases". Here we want to test tridesclous2 on 2 datasets and spykingcircus2 on one dataset
     # so it is a two level study (sorter_name, dataset)
     # this could be more complicated like (sorter_name, dataset, params)
     cases = {
@@ -74,20 +68,21 @@ The all mechanism is based on an intrinsic organization into a "study_folder" wi
             "label": "spykingcircus2 on tetrode0",
             "dataset": "toy0",
             "params": {
-                "sorter_name": "spykingcircus",
+                "sorter_name": "spykingcircus2",
                 "docker_image": True
             },
         },
     }
-    # this initilizes a folder
+    # this initializes a folder
+    study_folder = "my_study_folder"
     study = SorterStudy.create(study_folder=study_folder, datasets=datasets, cases=cases,
                                     levels=["sorter_name", "dataset"])
 
 
-    # This internally do run_sorter() for all cases in one function
+    # This internally does run_sorter() for all cases in one function
     study.run()
 
-    # Run the benchmark : this internanly do compare_sorter_to_ground_truth() for all cases
+    # Run the benchmark : this internally does compare_sorter_to_ground_truth() for all cases
     study.compute_results()
 
     # Collect comparisons one by one
@@ -104,9 +99,9 @@ The all mechanism is based on an intrinsic organization into a "study_folder" wi
         m = comp.get_confusion_matrix()
         w_comp = sw.plot_agreement_matrix(sorting_comparison=comp)
 
-    # Collect synthetic dataframes and display
+    # Collect synthetic dataframes and display.
     # As shown previously, the performance is returned as a pandas dataframe.
-    # The spikeinterface.comparison.get_performance_by_unit() function,
+    # The spikeinterface.comparison.get_performance_by_unit() function
     # gathers all the outputs in the study folder and merges them into a single dataframe.
     # Same idea for spikeinterface.comparison.get_count_units()
 
@@ -116,13 +111,10 @@ The all mechanism is based on an intrinsic organization into a "study_folder" wi
     # this is a dataframe
     unit_counts = study.get_count_units()
 
-    # Study also have several plotting methods for plotting the result
+    # Study also has several plotting methods for plotting the result
     study.plot_agreement_matrix()
     study.plot_unit_counts()
     study.plot_performances(mode="ordered")
-    study.plot_performances(mode="snr")
-
-
 
 
 Benchmark spike collisions

doc/modules/comparison.rst

@@ -1,7 +1,6 @@
 Comparison module
 =================
 
-
 SpikeInterface has a :py:mod:`~spikeinterface.comparison` module, which contains functions and tools to compare
 spike trains and templates (useful for tracking units over multiple sessions).
 
@@ -132,7 +131,7 @@ Given:
    4. **Compute performances**
 
       With the list of matched units we can compute performance metrics.
-      Given : **tp** the number of true positive events, **fp** number of false
+      Given: **tp** the number of true positive events, **fp** number of false
       positive events, **fn** the number of false negative events, **num_gt** the number
       of events of the matched tested units, the following metrics are computed for each GT unit:
 
@@ -142,7 +141,7 @@ Given:
         * false_discovery_rate = fp / (tp + fp)
         * miss_rate = fn / num_gt
 
-      The overall performances can be visualised with the **confusion matrix**, where
+      The overall performances can be visualized with the **confusion matrix**, where
       the last column contains the **FN** counts and the last row contains the **FP** counts.
 
     .. image:: ../images/spikecomparison_confusion.png
@@ -215,15 +214,16 @@ An **over-merged** unit has a relatively high agreement (>= 0.2 by default) for
 
 .. code-block:: python
 
-    local_path = download_dataset(remote_path='mearec/mearec_test_10s.h5')
-    recording, sorting_true = read_mearec(local_path)
+    from spikeinterface.widgets import plot_agreement_matrix, plot_confusion_matrix
+    from spikeinterface.generation import generate_ground_truth_recording
+    from spikeinterface.sorters import run_sorter
 
+    recording, sorting_true = generate_ground_truth_recording()
 
     # run a sorter and compare to ground truth
-    sorting_HS = run_sorter(sorter_name='herdingspike', recording=recording)
+    sorting_HS = run_sorter(sorter_name='herdingspikes', recording=recording)
     cmp_gt_HS = sc.compare_sorter_to_ground_truth(sorting_true, sorting_HS, exhaustive_gt=True)
 
-
     # To have an overview of the match we can use the ordered agreement matrix
     plot_agreement_matrix(cmp_gt_HS, ordered=True)
 
@@ -232,7 +232,6 @@ An **over-merged** unit has a relatively high agreement (>= 0.2 by default) for
     #
     perf = cmp_gt_HS.get_performance()
 
-
     # The confusion matrix is also a good summary of the score as it has
     # the same shape as an agreement matrix, but it contains an extra column for FN
     # and an extra row for FP
@@ -271,20 +270,20 @@ The :py:func:`~spikeinterface.comparison.compare_two_sorters()` returns the comp
 .. code-block:: python
 
     import spikeinterface as si
-    import spikeinterface.extractors as se
     import spikeinterface.sorters as ss
-    import spikeinterface.comparisons as sc
-    import spikinterface.widgets as sw
+    import spikeinterface.comparison as scmp
+    import spikeinterface.widgets as sw
 
     # First, let's generate a simulated dataset
     recording, sorting = si.generate_ground_truth_recording()
+
     # Then run two spike sorters and compare their outputs.
     sorting_HS = ss.run_sorter(sorter_name='herdingspikes', recording=recording)
     sorting_TDC = ss.run_sorter(sorter_name='tridesclous', recording=recording)
 
     # Run the comparison
     # Let's see how to inspect and access this matching.
-    cmp_HS_TDC = sc.compare_two_sorters(
+    cmp_HS_TDC = scmp.compare_two_sorters(
         sorting1=sorting_HS,
         sorting2=sorting_TDC,
         sorting1_name='HS',
@@ -339,7 +338,7 @@ Comparison of multiple sorters uses the following procedure:
     sorting_TDC = ss.run_sorter(sorter_name='tridesclous', recording=recording)
 
     # Compare multiple spike sorter outputs
-    mcmp = sc.compare_multiple_sorters(
+    mcmp = scmp.compare_multiple_sorters(
         sorting_list=[sorting_MS4, sorting_HS, sorting_TDC],
         name_list=['MS4', 'HS', 'TDC'],
         verbose=True,
@@ -354,7 +353,7 @@ Comparison of multiple sorters uses the following procedure:
     print(mcmp.comparisons[('MS4', 'TDC')].get_matching())
 
     # The global multi comparison can be visualized with this graph
-    sw.plot_multicomp_graph(multi_comparison=mcmp)
+    sw.plot_multicomparison_graph(multi_comparison=mcmp)
 
     # Consensus-based method
     #

doc/modules/core.rst

@@ -76,9 +76,9 @@ with 16 channels:
     recording_slice_frames = recording.frame_slice(start_frame=0,
                                                    end_frame=int(10*sampling_frequency))
     # get new recording with the first 4 channels
-    recording_slice_chans = recording.channel_slice(channel_ids=channel_ids[:4])
+    recording_slice_chans = recording.select_channels(channel_ids=channel_ids[:4])
     # remove last two channels
-    recording_rm_chans = recording.remove_channels(channel_ids=channel_ids[-2:])
+    recording_rm_chans = recording.remove_channels(remove_channel_ids=channel_ids[-2:])
 
     # set channel grouping (assume we have 4 groups of 4 channels, e.g. tetrodes)
     groups = [0] * 4 + [1] * 4 + [2] * 4 + [3] * 4
@@ -89,13 +89,14 @@ with 16 channels:
     # sliced recordings as values
 
     # set times (for synchronization) - assume our times start at 300 seconds
+    num_samples = recording.get_num_samples()
     timestamps = np.arange(num_samples) / sampling_frequency + 300
     recording.set_times(times=timestamps, segment_index=0)
 
 **Note**:
 Raw data formats often store data as integer values for memory efficiency. To give these integers meaningful physical units (uV), you can apply a gain and an offset.
 Many devices have their own gains and offsets necessary to convert their data and these values are handled by SpikeInterface for its extractors. This
-is triggered by the :code:`return_in_uV` parameter in :code:`get_traces()`, (see above example), which will return the traces in uV.
+is triggered by the :code:`return_in_uV` parameter in :code:`get_traces()`, (see above example), which will return the traces in uV. Read more in our how to guide, :ref:`physical_units`.
 
 
 Sorting
@@ -118,7 +119,7 @@ with 10 units:
 .. code-block:: python
 
     unit_ids = sorting.unit_ids
-    num_channels = sorting.get_num_units()
+    num_units = sorting.get_num_units()
     sampling_frequency = sorting.sampling_frequency
 
     # retrieve spike trains for a unit (returned as sample indices)
@@ -217,6 +218,7 @@ is run again with one of the backends supplied.
 
 .. code-block:: python
 
+    from pathlib import Path
     # create a "processed" folder
     processed_folder = Path("processed")
 
@@ -266,8 +268,7 @@ The :code:`sorting_analyzer` object implements convenient functions to access th
 
     num_channels = sorting_analyzer.get_num_channels()
     num_units = sorting_analyzer.get_num_units()
-    sampling_frequency = sorting_analyzer.get_sampling_frequency()
-    # or: sampling_frequency = sorting_analyzer.sampling_frequency
+    sampling_frequency = sorting_analyzer.sampling_frequency
     total_num_samples = sorting_analyzer.get_total_samples()
     total_duration = sorting_analyzer.get_total_duration()
 
@@ -689,6 +690,7 @@ In this example, we create a recording and a sorting object from numpy objects:
 .. code-block:: python
 
     import numpy as np
+    from spikeinterface.core import NumpyRecording, NumpySorting
 
     # in-memory recording
     sampling_frequency = 30_000.
@@ -697,7 +699,10 @@ In this example, we create a recording and a sorting object from numpy objects:
     num_channels = 16
     random_traces = np.random.randn(num_samples, num_channels)
 
-    recording_memory = NumpyRecording(traces_list=[random_traces])
+    recording_memory = NumpyRecording(
+        traces_list=[random_traces]
+        sampling_frequency=sampling_frequency,
+    )
     # with more elements in `traces_list` we can make multi-segment objects
 
     # in-memory sorting
@@ -706,7 +711,7 @@ In this example, we create a recording and a sorting object from numpy objects:
     spike_trains = []
     labels = []
     for i in range(num_units):
-        spike_trains_i = np.random.randint(low=0, high=num_samples, size=num_spikes_unit)
+        spike_trains_i = list(np.random.randint(low=0, high=num_samples, size=num_spikes_unit))
         labels_i = [i] * num_spikes_unit
         spike_trains += spike_trains_i
         labels += labels_i
@@ -751,7 +756,7 @@ the new objects will be *views* of the original ones.
 
     # keep one channel of every tenth channel
     keep_ids = recording.channel_ids[::10]
-    sub_recording = recording.channel_slice(channel_ids=keep_ids)
+    sub_recording = recording.select_channels(channel_ids=keep_ids)
 
     # keep between 5min and 12min
     fs = recording.sampling_frequency
@@ -870,13 +875,15 @@ They are useful to make examples, tests, and small demos:
 
 .. code-block:: python
 
+    from spikeinterface.core import generate_recording, generate_sorting, generate_snippets
+
     # recording with 2 segments and 4 channels
     recording = generate_recording(num_channels=4, sampling_frequency=30000.,
                                    durations=[10.325, 3.5], set_probe=True)
 
     # sorting with 2 segments and 5 units
     sorting = generate_sorting(num_units=5, sampling_frequency=30000., durations=[10.325, 3.5],
-                               firing_rate=15, refractory_period=1.5)
+                               firing_rates=15, refractory_period_ms=1.5)
 
     # snippets of 60 samples on 2 channels from 5 units
     snippets = generate_snippets(nbefore=20, nafter=40, num_channels=2,
@@ -929,7 +936,8 @@ WaveformExtractor
 ^^^^^^^^^^^^^^^^^
 
 This is now a legacy object that can still be accessed through the :py:class:`MockWaveformExtractor`. It is kept
-for backward compatibility.
+for backward compatibility. You can convert a ``WaveformExtractor`` to a ``SortingAnalyzer``
+easily, :ref:`using this guide <tutorials/waveform_extractor_to_sorting_analyzer:From WaveformExtractor to SortingAnalyzer>`.
 
 The :py:class:`~spikeinterface.core.WaveformExtractor` class is the core object to combine a
 :py:class:`~spikeinterface.core.BaseRecording` and a :py:class:`~spikeinterface.core.BaseSorting` object.