Skip to content

Commit 25c2511

Browse files
committed
simplify input logic
1 parent 41fc046 commit 25c2511

6 files changed

Lines changed: 699 additions & 202 deletions

File tree

common/src/main/scala/org/apache/comet/codegen/CometBatchKernelCodegen.scala

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,16 @@ object CometBatchKernelCodegen extends Logging with CometExprTraitShim {
121121
// `Unevaluable` but never touched by codegen (e.g. Spark 4.0's `ResolvedCollation`, which
122122
// lives in `Collate.collation` as a type marker; `Collate.genCode` delegates to its child).
123123
//
124+
// TODO(hof-lambdas): the `CodegenFallback` rule rejects `NamedLambdaVariable`, which flags
125+
// every higher-order function (`ArrayTransform`, `ArrayAggregate`, `ArrayExists`,
126+
// `ArrayFilter`, `ZipWith`, `MapFilter`, etc.) as unsupported. The variable is `CodegenFallback`
127+
// only in isolation; the surrounding HOF binds its `value` field inline as part of its own
128+
// `doGenCode`, and the resulting Java compiles fine. Loosening this would unlock
129+
// element-iteration over `Array<Struct>` / `Array<Map>` which today have no fuzz path
130+
// (`array_max` doesn't apply to non-comparable elements, generators are blocked above). Plan:
131+
// allow `NamedLambdaVariable` / `LambdaFunction` in the rejection scan; verify the kernel
132+
// splices the HOF's emitted loop without ctx.references collisions on the lambda holder.
133+
//
124134
// Nondeterministic / stateful expressions are accepted: per-partition kernel allocation
125135
// (`CometScalaUDFCodegen.ensureKernel`) plus a single `init(partitionIndex)` call at
126136
// partition entry give `Rand` / `MonotonicallyIncreasingID` / etc. correct state across

0 commit comments

Comments
 (0)