Question about the behaviour of partial_ratio with spaces

[x27lu-baml]

Hello, I noticed a weird behaviour with partial_ratio and comparing two pieces of text where one has spaces. For some reason, it will sometimes generate a score that is lower than it should be, given that it is doing sub-string comparison. I was able to notice this with exact text I have used here:

fuzz.partial_ratio("FG Kissimmee", " YM Kissimmee ") # output: 83.33333333333334

This returns a score of 83.333, instead of the best possible answer which to my understanding should be 90.909, which can be produced when I strip out the leading and trailing spaces for the text on the right:

fuzz.partial_ratio("FG Kissimmee", "YM Kissimmee") # output: 90.9090909090909

Additionally, it does not happen in all cases when spacing is applied, for example:

fuzz.partial_ratio("Hello World", " Hella World ") # output: 90.9090909090909
fuzz.partial_ratio("Hello World", "Hella World") # output: 90.9090909090909

Is this a bug, or is this an intentional behaviour?

[maxbachmann]

The reason is that partial ratio uses a sliding window of the shorter text on top of the longer text. So unless the substring is at start/end of the longer string it has the same length as the shorter string. It then returns the score for the alignment with the best fuzz.ratio.

This leads to surprising results in your case because for
`fuzz.partial_ratio("FG Kissimmee", " YM Kissimmee ")

it has to return
`fuzz.ratio("FG Kissimmee", "YM Kissimmee") == 83`

For `fuzz.partial_ratio("FG Kissimmee", "YM Kissimmee")` on the other hand the window starts / ends with the string so it can use a shorter substring
`fuzz.ratio("FG Kissimmee", " Kissimmee") == 90`

You can see this by running

fuzz.partial_ratio_alignment("FG Kissimmee", "YM Kissimmee")
ScoreAlignment(score=90.9090909090909, src_start=0, src_end=12, dest_start=2, dest_end=12)
fuzz.partial_ratio_alignment("FG Kissimmee", " YM Kissimmee ")
ScoreAlignment(score=83.33333333333334, src_start=0, src_end=12, dest_start=8, dest_end=20)

Now what could be done about this depends on the use case:
- we could add an option to disallow these shorter substrings
- if you want something that just finds an optimal alignment no matter the length but can cut of characters at start/end Smith Waterman might be a good option. Unfortunately I never got around to adding it to rapidfuzz (https://github.com/rapidfuzz/RapidFuzz/issues/175) even though a naive implementation should be fairly straight forward.

[x27lu-baml]

Hello, thanks for the quick reply! I'll figure out a workaround for this for now.

However, you brought up another point that I wanted to ask you about - is it possible to specify the window length to use for the search?

In the original code I had before using rapidfuzz, I was trying to figure out the lowest text distance comparison between a search text and a longer length of text to search in, for a given window size. I was iterating through the substrings of the longer text in python like below, which is inefficient (hence the motivation for replacement), but I had control over the search window, which I set to always be the length of the search text:

min_distance = sys.maxsize
for i in range(0, max(1, len(longer_text_to_search_in) - len(search_text) + 1)):
	distance = Levenshtein.distance(search_text, longer_text_to_search_in[i:i + len(search_text)])
	if distance < min_distance:
		min_distance = distance

partial_ratio replaces this implementation, but because I can't specify the window size, it is not an exact replacement, although for almost all cases in our testing it didn't make a difference to the results.

[maxbachmann]

we could add an option to disallow these shorter substrings

That I mentioned would do precisely that. It would enforce that the substring is always the same length as the search text. Since this would simply disable part of the processing it would be pretty straight forward to introduce this as option.