Version Upgrade Warnings

[Antorhythms]

UserWarning: Versions are not the same. This might lead to compatibility errors. Using spikeinterface==0.103.2 is recommended.

Upgrading to a new version of spike interface to take advantage of new fixes on recordings saved with a previous version (in my case 0.103.2 --> 0.104.1) triggered this warning.

In some sense I understand version changes might lead to changes in defaults and so on but this type of warning does not really help a user that upgraded for a reason. Was there really a breaking chnage between these two versions? If so could the warning be more targeted?

[chrishalcrow]

Good question. There are a small number of breaking changes: a metric name was renamed (but the name is updated on load) and a default parameter for snr was modified. But these are localised to the quality metrics extension. I'm sure there are others that are bug fixes, but nothing super serious.

I guess we should ask: what do we want the user to do when they see this warning? Check the release notes? Maybe we should only trigger the warning if there is something we consider a major breaking change between versions. Or maybe only trigger between major releases (e.g. 103 -> 104) and keep a page of analyzer-specific breaking changes that we link to in the warning.

What do you think @Antorhythms ? More opinions welcome from anyone else!

[h-mayorquin]

I think warnings should be informative and, as much as possible, actionable with a clear path to remove them. The current message asks users to downgrade, but if the object loads successfully it is probably fine. The real risk is subtle differences in default parameters or behavior across versions, not outright breakage.

So the warning should convey two things: (1) handle with care, there may be minor differences, and (2) to make the warning disappear, re-save the object with save(), save_to_folder(), or save_to_zarr(), which stamps the current version.

I have a PR that updates the message along these lines. What do you think?

[h-mayorquin]

#4505

[Antorhythms]

Thank you both for your replies!

In my case the warning was raised when loading a recording from a .pkl file saved with a previous version. This comes up naturally, for example, when you need to compute a new extension on a recording that was processed a while back. I think the pickled recording should reconstruct identically regardless of the spikeinterface version, since the pickle just stores the class and its constructor arguments, right? As long as the constructor signature hasn't changed the result is the same object. If that's correct, the warning is unnecessary when loading a recording.

Where version differences genuinely matter is in metric defaults and naming, things like peak_sign changing for SNR, or isolation_distance/l_ratio being merged into mahalanobis. But these are already handled by a separate backward-compatibility system on the analyzer extensions (_handle_backward_compatibility_on_load), and they only affect metric computation or param loading within the SortingAnalyzer. Hopefully this type of changes should be limited as well.

I would suggest

1. Not to warn when loading a recording (or any extractor) that has no actual breaking changes between versions.
2. Surface version-related warnings closer to where they matter — e.g. when computing metrics with changed defaults.

This way compatibility issues (metric renames, changed defaults) are flagged where the user can actually act on them rather than just prompting the user to resvae their recording to suppress a warning that is not fully explained.

Happy to hear more thoughts

[h-mayorquin]

You're right that the actual incompatibilities live in the analyzer extensions, not at the extractor level, and that _handle_backward_compatibility_on_load already covers the known cases.

Warning only where it matters (per-class, per-version-pair) is conceptually the right thing, but in practice it means maintaining a matrix of which classes break between which versions. That's a lot of upkeep for marginal gain, and it still wouldn't cover things we fail to anticipate. The value of the warning is precisely that safety net: letting users know that something might behave differently because of changes we didn't track explicitly.

With the updated message in #4505 the warning is honest about what it means ("there may be minor differences in defaults or behavior") and gives a clean actionable step to remove it (re-save the object). That keeps the cost low for us the maintainers and gives users a concrete path to get rid of the message.

[alejoe91]

Thanks guys. I think that @Antorhythms has a point though: that warning is annoying and I don't think we should suggest to re-save with any version update. That's why we added the _handle_backward_compatibility_on_load.

Maybe we could move the warning to the case where the extractor is explicitly handling some changes? (here)

[h-mayorquin]

Fair enough, I can't think from the top of my head where incompatibilities will arise for recordings as @Antorhythms says. Maybe we can remove it on recordings.