Commit d4e629f
authored
fix: Preserve quoted mixed-case identifiers in the
## Summary
This PR fixes a bug in the
`datafusion-examples/examples/relation_planner/pivot_unpivot.rs`
example implementation.
The example planner rewrites `PIVOT` into a `GROUP BY` plus `CASE`
expressions. During that
rewrite, it rebuilt column references using unquoted `col("...")`
expressions. That loses the
original identifier quoting and case-sensitivity, which breaks queries
that pivot on quoted
mixed-case columns such as `"pointNumber"`.
This PR preserves the original parsed column expression instead of
reconstructing it from a plain
string.
## Problem
Consider a query like:
```sql
SELECT *
FROM point_stats
PIVOT (
MAX(max_value)
FOR "pointNumber" IN ('16951' AS p16951, '16952' AS p16952)
)
ORDER BY ts
```
Before this change, the example planner:
1. Parsed `"pointNumber"` correctly as a quoted, case-sensitive
identifier.
2. Extracted its name as `pointNumber`.
3. Reconstructed new expressions with `col(&pivot_col_name)` and
`col(...)` for `GROUP BY`.
That reconstruction treated the identifier as an unquoted column
reference, which could be
normalized differently from the original schema field. In practice, this
means the planner could
end up looking for `pointnumber` while the schema still contained
`"pointNumber"`.
## Root Cause
Two places in the example rewrite logic rebuilt column references from
bare strings:
1. The generated `CASE` expression for the pivot column
2. The inferred `GROUP BY` expressions
That is fine for simple lowercase identifiers, but it is not correct for
quoted identifiers or
qualified fields because the reconstructed expression no longer carries
the original identifier
semantics.
## Fix
The fix is minimal:
1. Reuse the already planned `pivot_col` expression when building the
`CASE` expression.
2. Build `GROUP BY` expressions directly from the input schema via
`Expr::from(...)` rather than
re-creating them with `col(field_name)`.
This preserves:
- quoted mixed-case identifiers
- qualifiers
- original field resolution semantics
## Code Changes
File changed:
- `datafusion-examples/examples/relation_planner/pivot_unpivot.rs`
Main changes:
- Replace:
```rust
case(col(&pivot_col_name))
```
with:
```rust
case(pivot_col.clone())
```
- Replace string-based `GROUP BY` reconstruction:
```rust
schema
.fields()
.iter()
.map(|f| f.name().as_str())
...
.map(col)
```
with schema-derived expressions:
```rust
schema
.iter()
.filter(...)
.map(Expr::from)
```
## Example Coverage
This PR also adds an additional example scenario to the same file:
- a `point_stats` input table
- a `PIVOT` query using quoted mixed-case column `"pointNumber"`
- a snapshot asserting the expected output
This makes the bug and the fix visible directly in the example itself.pivot_unpivot example (#21432)1 parent 374806c commit d4e629f
File tree
1 file changed
+53
-5
lines changed- datafusion-examples/examples/relation_planner
1 file changed
+53
-5
lines changedLines changed: 53 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
217 | 217 | | |
218 | 218 | | |
219 | 219 | | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
220 | 239 | | |
221 | 240 | | |
222 | 241 | | |
| |||
288 | 307 | | |
289 | 308 | | |
290 | 309 | | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
291 | 338 | | |
292 | 339 | | |
293 | 340 | | |
| |||
415 | 462 | | |
416 | 463 | | |
417 | 464 | | |
418 | | - | |
419 | 465 | | |
420 | | - | |
421 | | - | |
422 | | - | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
423 | 471 | | |
424 | 472 | | |
425 | 473 | | |
| |||
434 | 482 | | |
435 | 483 | | |
436 | 484 | | |
437 | | - | |
| 485 | + | |
438 | 486 | | |
439 | 487 | | |
440 | 488 | | |
| |||
0 commit comments