ENH - Add configurable distance metric for fuzzy join nearest‑neighbor matching by sabasiddique1 · Pull Request #1861 · skrub-data/skrub

sabasiddique1 · 2026-01-26T17:14:10Z

REF: #1847
Goal
Allow users to choose the distance metric used for nearest‑neighbor matching in fuzzy_join / Joiner, enabling alternatives like cosine or manhattan for different data types.

Context / Issue
Addresses enhancement request to support non‑Euclidean distance metrics for fuzzy join nearest neighbors.

What changed (step‑by‑step)

Added a metric parameter to fuzzy_join and Joiner with default 'euclidean'.
Passed the metric through the matching classes to sklearn.neighbors.NearestNeighbors.
Documented the new parameter in docstrings with common metric examples.
Added tests covering multiple metrics and invalid metric handling.

Tests

pytest skrub/tests/test_fuzzy_join.py -v

MarieSacksick · 2026-01-28T08:59:33Z

hello @sabasiddique1!
Welcome :).

The CI doesn't pass, could you fix it please? It looks like it's because of the styling, it's explained how to deal with it here in the contribution guide.
In the mean time, you can put your PR in draft mode, so that it's clear that it's ongoing work. You can do it at the top right of the PR menu:

(I assumed it was still ongoing to I didn't have a look at the full code changes, but I still put a couple of comments on the tests in hope that it will alleviate the number of back and forth to iterate on the PR)

MarieSacksick · 2026-01-28T09:03:47Z


    c = fuzzy_join(b, a, left_on="col3", right_on="col1", add_match_info=True)
    assert ns.shape(c)[0] == len(b)
+def test_fuzzy_join_distance_metrics(df_module):


instead of repeating the code, you can parameterize it, for instance as it's done here or here.
Also, we want to test the values and not only the shape, to make sure that the input "metric" is taken into account, and that the function doesn't always use the default one.

Got it, I’ll parameterize the metric cases and assert the actual matched values (not just shape) so we verify that the metric parameter changes the result.

This comment was marked as resolved, but the actual problem was not addressed

MarieSacksick · 2026-01-28T09:04:29Z

+        left, right, on="A", suffix="r", metric='euclidean', add_match_info=False
+    )
+    assert ns.shape(result_euclidean)[0] == 2
+    assert ns.shape(result_euclidean)[1] == 3  # A, Ar, Br


The comment # A, Ar, Br is unclear to me, what do you mean here?

Thanks for pointing that out, I’ll remove or reword the ‘# A, Ar, Br’ comment for clarity.

This comment was also resolved without addressing the problem

rcap107

Thanks a lot for the PR, @sabasiddique1

I wrote some additional early comments to add to what @MarieSacksick has already mentioned

sabasiddique1 · 2026-01-30T16:38:45Z

Can we add the ‘no changelog needed’ label for this PR, or should I add file?

rcap107 · 2026-01-30T17:08:36Z

Can we add the ‘no changelog needed’ label for this PR, or should I add file?

There should be an entry in the "Changes" section of the changelog. All user-facing changes should be reported, and adding a parameter is a user-facing change.

rcap107

Hi @sabasiddique1, there are still a few things to fix. Given the other comments, I thought some of the point had already been addressed. Maybe you forgot to commit your changes?

rcap107 · 2026-02-12T14:29:42Z


    c = fuzzy_join(b, a, left_on="col3", right_on="col1", add_match_info=True)
    assert ns.shape(c)[0] == len(b)
+def test_fuzzy_join_distance_metrics(df_module):


This comment was marked as resolved, but the actual problem was not addressed

rcap107 · 2026-02-12T14:30:02Z

+        left, right, on="A", suffix="r", metric='euclidean', add_match_info=False
+    )
+    assert ns.shape(result_euclidean)[0] == 2
+    assert ns.shape(result_euclidean)[1] == 3  # A, Ar, Br


This comment was also resolved without addressing the problem

rcap107 · 2026-02-12T14:31:08Z

+    assert ns.shape(result_euclidean)[1] == 3  # A, Ar, Br
+
+    # Test with cosine
+    result_cosine = fuzzy_join(


The actual value of the result should be tested against the expected value as a sanity check, for this and all the other metrics.

rcap107 · 2026-02-12T14:32:28Z

+    (using the specified metric) computed to find, for each main table row, its
+    nearest neighbor within the auxiliary table.
+
+    Insert some columns whose names start with skrub_Joiner containing


This section of the docstring looks like an incorrect copypaste, why is it here?

rcap107 · 2026-02-12T14:34:10Z

  :class:`TfidfVectorizer`.
  :pr:`1819` by :user:`Eloi Massoulié <emassoulie>`
+- Added a ``metric`` parameter to :func:`fuzzy_join` and :class:`Joiner` to configure
+  the nearest-neighbor distance used for matching.


Please mention that metric can be one of the metrics supported by sklearn.neighbors.NearestNeighbors, with a reference to the its docstring.

…61-1

rcap107 · 2026-04-14T13:21:57Z

Hi @sabasiddique1, I've updated the pr manually to avoid trouble with possible merge conflicts. I'm going to review this again today (or tomorrow at the latest). After that we can see if there is still work to do, or if it can be merged.

rcap107

The PR looks good to me, thanks a lot @sabasiddique1!

I'll leave this open for a few more days in case other maintainers want to take a look, then I'll merge it.

rcap107 · 2026-04-23T07:48:12Z

Merging this, thanks @sabasiddique1

MarieSacksick reviewed Jan 28, 2026

View reviewed changes

Comment thread skrub/tests/test_fuzzy_join.py Outdated

MarieSacksick reviewed Jan 28, 2026

View reviewed changes

rcap107 requested changes Jan 29, 2026

View reviewed changes

Comment thread skrub/_fuzzy_join.py Outdated

Comment thread skrub/_fuzzy_join.py Outdated

Comment thread skrub/_joiner.py Outdated

Comment thread skrub/_matching.py Outdated

rcap107 mentioned this pull request Jan 29, 2026

DOC: explain unpacking results from deferred functions #1870

Merged

rcap107 changed the title ~~Add configurable distance metric for fuzzy join nearest‑neighbor matching~~ ENH - Add configurable distance metric for fuzzy join nearest‑neighbor matching Jan 29, 2026

rcap107 linked an issue Jan 29, 2026 that may be closed by this pull request

ENH - Allow using a different distance for the nearest neighbors in fuzzy join #869

Closed

sabasiddique1 marked this pull request as draft January 30, 2026 14:16

sabasiddique1 force-pushed the feat-distance-metric-fuzzy-join branch from 040157a to 1a599ab Compare January 30, 2026 16:36

sabasiddique1 marked this pull request as ready for review January 30, 2026 17:03

rcap107 added this to the Release 0.8.0 milestone Feb 9, 2026

rcap107 reviewed Feb 12, 2026

View reviewed changes

sabasiddique1 added 4 commits February 14, 2026 14:19

Add distance metric option to fuzzy join

e38ff1a

Update fuzzy_join metric tests and docstrings

64f4ee1

Update fuzzy_join metric tests and docs

4cc13ad

docs: add changelog entry for fuzzy-join metric

bccc4d9

sabasiddique1 force-pushed the feat-distance-metric-fuzzy-join branch from 07646ba to bccc4d9 Compare February 15, 2026 11:06

rcap107 modified the milestones: Release 0.8.0, Release 0.8.1 Mar 18, 2026

rcap107 added 2 commits April 14, 2026 15:17

Merge remote-tracking branch 'upstream/HEAD' into pr/sabasiddique1/18…

cd54a07

…61-1

changelog

b45f488

rcap107 approved these changes Apr 15, 2026

View reviewed changes

rcap107 merged commit bc2c689 into skrub-data:main Apr 23, 2026
29 checks passed

Conversation

sabasiddique1 commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MarieSacksick commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rcap107 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sabasiddique1 commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rcap107 commented Jan 30, 2026

Uh oh!

rcap107 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rcap107 commented Apr 14, 2026

Uh oh!

rcap107 left a comment

Choose a reason for hiding this comment

Uh oh!

rcap107 commented Apr 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sabasiddique1 commented Jan 26, 2026 •

edited

Loading

MarieSacksick commented Jan 28, 2026 •

edited

Loading

rcap107 left a comment •

edited

Loading

sabasiddique1 commented Jan 30, 2026 •

edited

Loading