Skip to content

Commit d19e1f2

Browse files
authored
docs: add guidance on disabling constant folding for literal tests (#3200)
1 parent 395f7de commit d19e1f2

1 file changed

Lines changed: 23 additions & 0 deletions

File tree

docs/source/contributor-guide/adding_a_new_expression.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,29 @@ test("unhex") {
236236
}
237237
```
238238

239+
#### Testing with Literal Values
240+
241+
When writing tests that use literal values (e.g., `SELECT my_func('literal')`), Spark's constant folding optimizer may evaluate the expression at planning time rather than execution time. This means your Comet implementation might not actually be exercised during the test.
242+
243+
To ensure literal expressions are executed by Comet, disable the constant folding optimizer:
244+
245+
```scala
246+
test("my_func with literals") {
247+
withSQLConf(SQLConf.OPTIMIZER_EXCLUDED_RULES.key ->
248+
"org.apache.spark.sql.catalyst.optimizer.ConstantFolding") {
249+
checkSparkAnswerAndOperator("SELECT my_func('literal_value')")
250+
}
251+
}
252+
```
253+
254+
This is particularly important for:
255+
256+
- Edge case tests using specific literal values (e.g., null handling, overflow conditions)
257+
- Tests verifying behavior with special input values
258+
- Any test where the expression inputs are entirely literal
259+
260+
When possible, prefer testing with column references from tables (as shown in the `unhex` example above), which naturally avoids the constant folding issue.
261+
239262
### Adding the Expression To the Protobuf Definition
240263

241264
Once you have the expression implemented in Scala, you might need to update the protobuf definition to include the new expression. You may not need to do this if the expression is already covered by the existing protobuf definition (e.g. you're adding a new scalar function that uses the `ScalarFunc` message).

0 commit comments

Comments
 (0)