Skip to content

feat: push IN predicate down to Lance filter engine#21

Open
wombatu-kun wants to merge 1 commit into
lance-format:mainfrom
wombatu-kun:feature/in-pushdown
Open

feat: push IN predicate down to Lance filter engine#21
wombatu-kun wants to merge 1 commit into
lance-format:mainfrom
wombatu-kun:feature/in-pushdown

Conversation

@wombatu-kun
Copy link
Copy Markdown

@wombatu-kun wombatu-kun commented Apr 21, 2026

Summary

  • Implement IN predicate pushdown in LanceDynamicTableSource (previously returned null, forcing Flink to evaluate IN in-process).
  • Expose getFilters() so tests can verify the rendered Lance SQL.

Test plan

  • mvn test — 178/178 passing (17 skipped = MinIO integration gate, no env vars).
  • LanceReadOptimizationsTest#testInPredicatePushDown now asserts both acceptance and the exact string status IN ('active', 'pending', 'completed').

🤖 Generated with Claude Code

@wombatu-kun wombatu-kun force-pushed the feature/in-pushdown branch from 9eac2c7 to 714f3b6 Compare May 24, 2026 08:37
@github-actions github-actions Bot added the enhancement New feature or request label May 24, 2026
@claude
feat: push IN predicate down to Lance filter engine
5901c26 
Render `field IN (v1, v2, ...)` from Flink's IN CallExpression
using the existing literal-escaping helper, and expose getFilters()
so tests can assert the exact Lance SQL string.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wombatu-kun wombatu-kun force-pushed the feature/in-pushdown branch from 714f3b6 to 5901c26 Compare May 24, 2026 09:04
@wombatu-kun wombatu-kun mentioned this pull request May 24, 2026
Open
@wombatu-kun
Copy link
Copy Markdown
Author

wombatu-kun commented May 24, 2026

This PR doubles as a fix for a pre-existing failure on main: LanceReadOptimizationsTest.testInPredicatePushDown already lives on main and asserts IN-predicate pushdown, but the implementation (buildInFilter) is only added here. So main's test suite is currently red on that test, and any test-running CI (e.g. #25) stays red until this lands — merging #21 first turns it green.

@wombatu-kun
Copy link
Copy Markdown
Author

wombatu-kun commented Jun 5, 2026

This PR doubles as a fix for a pre-existing failure on main: LanceReadOptimizationsTest.testInPredicatePushDown already lives on main and asserts IN-predicate pushdown, but the implementation (buildInFilter) is only added here. So main's test suite is currently red on that test, and any test-running CI (e.g. #25) stays red until this lands — merging #21 first turns it green.

hi @fightBoxing review and merge it please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@wombatu-kun