Commit 2d83237
committed
[SPARK-57180][SQL] Skip statically-dead setNullAt branch in GenerateSafeProjection for non-nullable fields
### What changes were proposed in this pull request?
`GenerateSafeProjection` emits `if (isNull) { mutableRow.setNullAt(i); } else { convert; setColumn; }` for every projected field. When the field expression is statically non-nullable, `isNull` is `FalseLiteral`, so the `setNullAt` branch is dead and only the `else` ever runs. This patch detects that case (`evaluationCode.isNull == FalseLiteral` -- the idiom already used by `GenerateUnsafeProjection`) and emits just the conversion + `setColumn`. The nullable path is unchanged. (`GenerateMutableProjection` already branches on `e.nullable`; this brings `GenerateSafeProjection` in line.)
### Why are the changes needed?
Part of SPARK-56908 (umbrella). Removes a dead branch (and the `setNullAt` call) per non-nullable field from the generated safe projection, shrinking the emitted Java for wide non-nullable schemas (helping with the JVM 64KB method / constant-pool limits, Janino compile time, and JIT work).
### Does this PR introduce _any_ user-facing change?
No. The compiled behavior is identical; only the emitted Java source text changes.
### How was this patch tested?
Added `SPARK-57180: SafeProjection over statically non-nullable fields` to `GeneratedProjectionSuite`, projecting non-nullable int/string/struct/array fields through an unsafe -> safe round trip and asserting the values.
```
build/sbt "catalyst/testOnly *GeneratedProjectionSuite" # 10/10
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8)
Closes #56231 from gengliangwang/spark-safeprojection-deadnull-codegen.
Authored-by: Gengliang Wang <gengliang@apache.org>
Signed-off-by: Gengliang Wang <gengliang@apache.org>1 parent 6b84c98 commit 2d83237
2 files changed
Lines changed: 39 additions & 3 deletions
File tree
- sql/catalyst/src
- main/scala/org/apache/spark/sql/catalyst/expressions/codegen
- test/scala/org/apache/spark/sql/catalyst/expressions/codegen
Lines changed: 12 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
158 | | - | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
159 | 167 | | |
160 | 168 | | |
161 | 169 | | |
162 | 170 | | |
163 | | - | |
164 | | - | |
| 171 | + | |
165 | 172 | | |
166 | 173 | | |
| 174 | + | |
| 175 | + | |
167 | 176 | | |
168 | 177 | | |
169 | 178 | | |
| |||
Lines changed: 27 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
214 | 241 | | |
215 | 242 | | |
216 | 243 | | |
| |||
0 commit comments