feat: add Pyversity integration

[kacperlukawski]

Related Issues

• fixes Add Pyversity integration #2858

Proposed Changes:

Integrate Pyversity for retrieval results diversification.

How did you test it?

Unit tests are included similarly to the other integrations.

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[sjrl]

@kacperlukawski thanks for the new integration! A few high level comments I want to provide as feedback:

• We need to add to_dict and from_dict methods to this new component otherwise it won't be possible to configure it via yaml which we use in our cloud platform.
• I was also wondering is there value in letting k be set to None? My thought would be if set to None then all documents should be returned just reranked using the diversify algorithm. I believe this something we already do in our other rankers as well. What do you think?

[kacperlukawski]

@kacperlukawski thanks for the new integration! A few high level comments I want to provide as feedback:

• We need to add to_dict and from_dict methods to this new component otherwise it won't be possible to configure it via yaml which we use in our cloud platform.
• I was also wondering is there value in letting k be set to None? My thought would be if set to None then all documents should be returned just reranked using the diversify algorithm. I believe this something we already do in our other rankers as well. What do you think?

Thanks for having a look @sjrl! I added to_dict and from_dict, and allowed setting top_k=None. I noticed the other rerankers modify the score of the documents, so it's also implemented accordingly.

[sjrl]

Do you think it would also make sense to expose some of the other params like top_k, strategy or diversity at runtime as well? We often have top_k as a runtime param as well so users can easily test without needing to reinitialize the component or pipeline.

[kacperlukawski]

Yes, makes sense if we do that for other components. Let me extend it.

[sjrl]

Looks good! Up to you if you'd like to expand the run method. We can always do it later if users request it as a feature.

[kacperlukawski]

@sjrl I decided to add a way to override all these params in runtime and some tests to cover these cases.

[sjrl]

@sjrl I decided to add a way to override all these params in runtime and some tests to cover these cases.

Looks good, feel free to merge!