feat: make polars-lazy optional behind 'lazy' feature#12
Open
0xDaizz wants to merge 7 commits into
Open
Conversation
…ness fixes - engine: collect_gpu를 lf._collect_post_opt() 콜백 기반으로 재구현하여 polars 내부 코드 변경 없이 GPU 실행 파이프라인 연결 (IR → DataFrameScan 치환) - expr: First/Last agg를 slice+repeat 방식으로 수정하여 올바른 단일값 브로드캐스트 보장, 빈 컬럼/그룹 guard 추가 (null_column_for_type 공통화) - column: try_data_type() 추가 — data_type()의 fallible 버전으로 FFI type_id 불일치 시 panic 대신 Result 반환 - convert: Arrow FFI ptr::read 4곳에 debug_assert_eq 크기 검증 + SAFETY 주석 보강 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Not expression: Bool→logical not, Int→bitwise invert (Polars semantics) - IsIn expression: values.contains + nulls_equal propagation (null in col → null output when nulls_equal=false) - GroupBy Quantile: extract q via DynLiteralValue/Scalar for Polars 0.53 LiteralValue API (Dyn/Scalar/Series/Range variants) - Enable `is_in` feature in polars-plan dependency - Codex rescue review 3건 반영: LiteralValue variant mismatch, Not integer dispatch, IsIn null semantics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
engine.rs: Replace Vec::position() linear scan with HashMap for column name lookup in HStack, eliminating O(n²) behavior on wide DataFrames. expr.rs: Replace host-allocated vec![v; height] + from_slice with cudf::filling::sequence GPU-native constant fill for Int32/Int64/ Float32/Float64 scalar broadcasts, avoiding host→device round-trip. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Date/Datetime/Duration type mapping in types.rs with explicit rejection of timezone-aware Datetime. Support temporal null columns (i32 for DurationDays/TimestampDays, i64 for all other temporal types) and scalar broadcast pass-through. Add temporal pass-through in arithmetic_output_type to let cudf handle native type promotion. Incorporates fixes from Codex rescue (5 blocking issues resolved). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implement per-row broadcast for window aggregations via groupby→left_join→ sort_by_key pipeline. Left join output order is not guaranteed by libcudf, so sort by left_indices to restore original row alignment. - Add AExpr::Over handler with GroupsToRows dispatch - Explicitly reject order_by (not yet supported) - Promote extract_agg_info/map_ir_agg to pub(crate) for reuse Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This breaks the circular dependency when polars-lazy depends on cudf-polars for GPU engine dispatch. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…oral types, performance optimizations Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
polars-lazydependency optional in cudf-polars, gated behindlazyfeaturecollect_gpufunction is only available whenlazyfeature is enabledTest plan
cargo check -p cudf-polars --no-default-featuresshould compile without polars-lazycargo check -p cudf-polars --features lazyshould compile with polars-lazy🤖 Generated with Claude Code