Document z_score normalization and rrf combination in Hybrid Optimizer#12483
Document z_score normalization and rrf combination in Hybrid Optimizer#12483RexyZhang2001 wants to merge 2 commits into
Conversation
Signed-off-by: Rex Zhang <yichenzg@amazon.com>
RexyZhang2001
requested review from
cwperks,
dlvenable,
epugh,
kolchfa-aws,
mgodwan,
peterzhuamazon and
sumobrian
as code owners
|
Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Merged. Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer. When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). |
kolchfa-aws
added
release-notes
| * Two normalization techniques: `l2` and `min_max`. | ||
| **Score-based variants**: | ||
|
|
||
| * Three normalization techniques: `l2`, `min_max`, and `z_score`. (`z_score` is only paired with `arithmetic_mean`.) |
There was a problem hiding this comment.
can we add link to z_score documentation (https://docs.opensearch.org/latest/search-plugins/search-pipelines/normalization-processor/#request-body-fields), fact that it's compatible with only am is not coming from SRW implementation but it's z_score level restriction
There was a problem hiding this comment.
Added in new commit
|
|
||
| **Rank-based variants**: | ||
|
|
||
| * The `rrf` (Reciprocal Rank Fusion) combination technique, evaluated with `rank_constant` values of `1`, `5`, `10`, `20`, and `60`. |
There was a problem hiding this comment.
please add link to score based processor page https://docs.opensearch.org/latest/search-plugins/search-pipelines/score-ranker-processor/. Worth mentioning that we are using default equal weights
There was a problem hiding this comment.
Added in new commit
Signed-off-by: Rex Zhang <yichenzg@amazon.com>
4 participants
Description
Updates the Hybrid Optimizer documentation to reflect two newly added techniques:
z_scorenormalizationrrf(Reciprocal Rank Fusion) combination, parameterized byrank_constantExisting language and structure are preserved; the bullet list is split into a Score-based variants group and a Rank-based variants group to accommodate RRF, which has different parameters.
Issues Resolved
Closes #12482
Version
3.7.0
Related PR
Checklist
For more information on following Developer Certificate of Origin and signing off your commits, please check here.