Commit 5d491f6
committed
revert [SPARK-43752] and add test
### What changes were proposed in this pull request?
This PR reverts the code changes from #55127 (commit eca57ea) and moves the test to `InsertIntoTests` where it properly runs across V2 INSERT SQL test suites.
Specifically:
- **Removes** the `V2WriteCommand` case added to `Analyzer.ResolveReferences` and `ResolveColumnDefaultInCommandInputQuery`
- **Removes** the test from `ResolveDefaultColumnsSuite` (wrong location for a V2 end-to-end test)
- **Adds** the test to `InsertIntoSQLOnlyTests` in `InsertIntoTests.scala`, which runs in suites with `includeSQLOnlyTests = true`: `DataSourceV2SQLSuite`, `DataSourceV2SQLSessionCatalogSuite`, and `V1WriteFallbackSessionCatalogSuite`. This is correct since `DEFAULT` is SQL-only syntax.
### Why are the changes needed?
The code added by #55127 is dead code. For `INSERT INTO v2_table VALUES (1, DEFAULT)`, the resolution flow is:
1. The parser produces `InsertIntoStatement` for **both** V1 and V2 tables.
2. In the Resolution batch, `ResolveReferences` dispatches `InsertIntoStatement` to `resolveColumnDefaultInCommandInputQuery`, which resolves DEFAULT using the table schema. For V2 tables, the `DataSourceV2Relation` schema includes column default metadata (via `CatalogV2Util.encodeDefaultValue`), so DEFAULT is resolved correctly at this stage.
3. `ResolveInsertInto` only converts `InsertIntoStatement` to a `V2WriteCommand` (`AppendData`/`OverwriteByExpression`/`OverwritePartitionsDynamic`) **after** the query is fully resolved — by which point DEFAULT is already gone.
No code path (SQL or DataFrame API) creates a `V2WriteCommand` with unresolved DEFAULT attributes. The `PlanResolutionSuite` test "INSERT INTO table with default column value" already verifies this resolution at the plan level.
The test is still valuable and passes without the code changes — this PR moves it to the proper location.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Moved the existing test to `InsertIntoSQLOnlyTests` in `InsertIntoTests.scala`. The test verifies:
- `INSERT INTO` with partial DEFAULT values
- `INSERT INTO` with all DEFAULT values
- `INSERT OVERWRITE` with DEFAULT values
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code 4.6
Closes #55348 from cloud-fan/SPARK-43752-revert.
Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>1 parent 6d2840a commit 5d491f6
4 files changed
Lines changed: 17 additions & 36 deletions
File tree
- sql
- catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis
- core/src/test/scala/org/apache/spark/sql
- connector
Lines changed: 0 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1418 | 1418 | | |
1419 | 1419 | | |
1420 | 1420 | | |
1421 | | - | |
1422 | | - | |
1423 | | - | |
1424 | | - | |
1425 | | - | |
1426 | | - | |
1427 | 1421 | | |
1428 | 1422 | | |
1429 | 1423 | | |
| |||
Lines changed: 0 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | 92 | | |
99 | 93 | | |
100 | 94 | | |
| |||
Lines changed: 0 additions & 24 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
308 | 308 | | |
309 | 309 | | |
310 | 310 | | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
317 | | - | |
318 | | - | |
319 | | - | |
320 | | - | |
321 | | - | |
322 | | - | |
323 | | - | |
324 | | - | |
325 | | - | |
326 | | - | |
327 | | - | |
328 | | - | |
329 | | - | |
330 | | - | |
331 | | - | |
332 | | - | |
333 | | - | |
334 | | - | |
335 | 311 | | |
Lines changed: 17 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
505 | 505 | | |
506 | 506 | | |
507 | 507 | | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
508 | 525 | | |
509 | 526 | | |
510 | 527 | | |
| |||
0 commit comments