Add IBL's "noise cutoff" quality metric

[huizizhang949]

Add a new metric "noise cutoff" based on IBL https://figshare.com/articles/online_resource/Spike_sorting_pipeline_for_the_International_Brain_Laboratory/19705522?file=49783080 in a new function compute_noise_cutoffs.

The original implementation is here https://github.com/int-brain-lab/ibllib/blob/2e1f91c622ba8dbd04fc53946c185c99451ce5d6/brainbox/metrics/single_units.py.

The "noise cutoff" metric assesses whether a unit’s spike‐amplitude distribution is truncated
at the low-end, which may be due to the high amplitude detection threhold in the deconvolution step,
i.e., if low‐amplitude spikes were missed. It does not assume a Gaussian shape;
instead, it directly compares counts in the low‐amplitude bins to counts in high‐amplitude bins (cutoff) and the peak bin (ratio).

Here are examples with no truncation and truncation, which give low and high values (cutoff and ratio), respectively.

I have added documentation and tests.

[zm711]

some initial comments, suggestions, questions :)

[zm711]

There is no discussion of what number of bins means for this metric. Might be worth adding. Why not do 10, or 1000 bins?

[huizizhang949]

I have added a comment to the docs. This is also the default value used in IBL's implementation.

[zm711]

Shouldn't this error occur right after peak sign so we don't waste time doing the _get_amplitudes_by_units function?

[zm711]

Also how does a user know what the peak sign is? Where did it come from. I would consider making this note refer to the peak sign of the spike_amplitudes extension.

[huizizhang949]

I added a comment about where to change the peak_sign in the error message. I moved the error before _get_amplitudes_by_units.

[zm711]

From a profiling perspective is there any benefit to thinking about doing a numba/numpy dual strategy? It seems like _noise_cutoff is all numpy based so could be sped up with a numba version. That would complicate the PR strategy, but would be curious about the profiling. It would also be an excellent exercise in profiling code performance. Definitely not required unless this is really slow. That's the reason we had to shut off the nn metrics as a default for qm stuff.

[samuelgarcia]

lets have in mind that this _get_amplitudes_by_units is not efficient because it internal use mask on all units.

amplitude_extension.get_data(outputs="by_units") is really faster for this. We should use it here and also inside
_get_amplitudes_by_units the only pain is that amplitude will be splited over segments but concatenate is fast.

[huizizhang949]

We just tried to implement amplitude_extension.get_data(outputs="by_units") and then concatenated, but it became very complex. We propose that there should be another PR to make _get_amplitudes_by_units to use the approach suggested.

[samuelgarcia]

Thank you for this, this is really nice.
This is a bit similar for the global question that "amplitude_cutoff" but with a differents approach and implementation.
The actual "amplitude_cutoff" is not super reliable.
This looks easier.
I am wondering how much this ratio is able to discriminate a real cut in the distribution like you have on the right plots and a distribution with a positive skewness coeeficient.

[zm711]

The actual "amplitude_cutoff" is not super reliable.

related to this, I don't think I've ever got amplitude_cutoff to work on my data. I always get nans for it. So this could be an interesting if it works more often.

[chrishalcrow]

Hello @oliche @GaelleChapuis , @huizizhang949 has implemented a version of the IBL noise cutoff metric here in spikeinterface (docs: https://spikeinterface--3984.org.readthedocs.build/en/3984/modules/qualitymetrics/noise_cutoff.html). Hoping it will be a helpful tool here!

If you have time to have a look at the implementation, we'd appreciate any feedback! The bulk of the code is in def _noise_cutoff.

[alejoe91]

@huizizhang949 why did you close this? I think it was in very good shape!