Skip to content

feat: support vector search/similarity as UDFs#80

Merged
ChunxuTang merged 3 commits into
lance-format:mainfrom
ChunxuTang:vector
Jan 4, 2026
Merged

feat: support vector search/similarity as UDFs#80
ChunxuTang merged 3 commits into
lance-format:mainfrom
ChunxuTang:vector

Conversation

@ChunxuTang

Copy link
Copy Markdown
Collaborator

This PR implements the basic vector search functionalities as datafusion UDFs. It supports queries like:

-- Simple k-NN search
MATCH (p:Person)
WHERE vector_distance(p.embedding, [0.1, 0.2], cosine) < 0.5
RETURN p.name
LIMIT 10

-- Order by similarity
MATCH (p:Person)
RETURN p.name, vector_distance(p.embedding, [0.1, 0.2], l2) AS dist
ORDER BY dist ASC
LIMIT 10

-- Hybrid: Property filter + vector search
MATCH (p:Person)
WHERE p.age > 25
  AND vector_similarity(p.embedding, [0.1, 0.2], cosine) > 0.8
RETURN p.name

See the test_vector_search.rs file for more query examples.

Meanwhile, it adds the parsing of parameters ($query_vector), so later we could also support passing in a parameter for comparison.

Note: Currently, it only works with in-memory data. We'll add the support of Lance ANN indexes as a following-up task.

@codecov-commenter

Copy link
Copy Markdown

@ChunxuTang ChunxuTang merged commit f7c5e20 into lance-format:main Jan 4, 2026
9 checks passed
@ChunxuTang ChunxuTang deleted the vector branch January 10, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants