Skip to content

Add _precomputable_kwarg_names to BaseExtractor#3781

Merged
samuelgarcia merged 6 commits into
SpikeInterface:mainfrom
chrishalcrow:add-precomputed-kwargs
Mar 26, 2025
Merged

Add _precomputable_kwarg_names to BaseExtractor#3781
samuelgarcia merged 6 commits into
SpikeInterface:mainfrom
chrishalcrow:add-precomputed-kwargs

Conversation

@chrishalcrow
Copy link
Copy Markdown
Member

As discussed in #3685 (comment) and in maintainer meetings, it would be handy to have a _precomputable_kwargs in the BaseExtractor, especially for Preprocessors.

These are kwargs which can be precomputed for use in a preprocessor. For instance, if you know the whitening matrices, you can pass these directly instead of computing them. If you know the bad channel ids, you can pass these to RemoveBadChannels (once it exists #3685) and if you know your motion object, you could pass this to a CorrectMotion class (once this exists too!).

This can also be used when saving preprocessed recordings. If we save the provenance and the precomputed data, we can load a preprocessed recording much faster. This will be implemented in a later PR.

This PR has no functionality so there are no new tests.

Questions:

  • Naming. @alejoe91 had suggested on _precomputed_kwargs but I think this is confusing since these kwargs aren't always precomputed, it's just that they can be precomputed. So I've gone for _precomputable_kwargs. Opinions?
  • Should this be in BaseExtractor? Could be in BasePreprocessor? Any use cases for it in scenarios other than preprocessing?

@chrishalcrow chrishalcrow added the core Changes to core module label Mar 18, 2025
@alejoe91
Copy link
Copy Markdown
Member

Thanks @chrishalcrow

I think that BaseExtractor is ok!

@alejoe91
Copy link
Copy Markdown
Member

@samuelgarcia ok to you?

Comment thread src/spikeinterface/core/base.py Outdated
self._kwargs = {}

# kwargs which can be precomputed before being used by the extractor
self._precomputable_kwargs = []
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree able is better if its possible but not necessarily done :)

@chrishalcrow
Copy link
Copy Markdown
Member Author

@samuelgarcia _precomputable_kwargs is now a class attribute.

@h-mayorquin
Copy link
Copy Markdown
Collaborator

My two cents:

  1. Kwargs is usually used (in this codebase as well) for dict objects. I think ._precomputable_kwargs is a good name but it should be a dict and follow kwarg conventions.
  2. I say isolate to BasePocessor and then generalize when and if we needed. It is good to keep complexity away from the core as long as we can.

I can't really say a lot because I have not seen it in use. Perhaps a use case other than #3781 is the estimated noise and std of recordings. We already have some mechanism to treat that as special quantities and maybe they should be unified with this.Also, how will this work with the json serialization is not clear to me. Maybe you discussed this on the meeting that I missed.

All of that said, these is a private attribute so I am not too concerned if we need to change it later.

@alejoe91
Copy link
Copy Markdown
Member

alejoe91 commented Mar 21, 2025

My two cents:

  1. Kwargs is usually used (in this codebase as well) for dict objects. I think ._precomputable_kwargs is a good name but it should be a dict and follow kwarg conventions.
  2. I say isolate to BasePocessor and then generalize when and if we needed. It is good to keep complexity away from the core as long as we can.

I can't really say a lot because I have not seen it in use. Perhaps a use case other than #3781 is the estimated noise and std of recordings. We already have some mechanism to treat that as special quantities and maybe they should be unified with this.Also, how will this work with the json serialization is not clear to me. Maybe you discussed this on the meeting that I missed.

All of that said, these is a private attribute so I am not too concerned if we need to change it later.

  1. What about _precomputable_kwarg_names?

  2. I vote for keeping it in base, because soon we will need some logic in the loading functions to either keep or discard precomputable kwargs.

@chrishalcrow
Copy link
Copy Markdown
Member Author

When we discussed, we agreed not to use this to store any kwargs themselves, just the information about which kwargs could be precomputed. So I think a list is the correct structure; very happy to call _precomputable_kwarg_names.

I'm indifferent to putting it in BaseExtractor or BaseProcessor.

@alejoe91
Copy link
Copy Markdown
Member

When we discussed, we agreed not to use this to store any kwargs themselves, just the information about which kwargs could be precomputed. So I think a list is the correct structure; very happy to call _precomputable_kwarg_names.

I'm indifferent to putting it in BaseExtractor or BaseProcessor.

Yes, but what @h-mayorquin is saying is that the name _precomputable_kwargs suggests that it contains kwargs (a dict).

@chrishalcrow chrishalcrow changed the title Add _precomputable_kwargs to BaseExtractor Add _precomputable_kwarg_names to BaseExtractor Mar 21, 2025
Copy link
Copy Markdown
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, we will see how it is used.

It could also be kwargs_keys to keep with the dict semantics but names is just fine if not more expressive.

@samuelgarcia
Copy link
Copy Markdown
Member

super

@samuelgarcia samuelgarcia merged commit 165035a into SpikeInterface:main Mar 26, 2025
@chrishalcrow chrishalcrow deleted the add-precomputed-kwargs branch July 30, 2025 10:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Changes to core module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants