Commit 2df302d
[SPARK-56607][PYTHON][FOLLOWUP] Use pyspark.sql.DataFrame to support connect-only
### What changes were proposed in this pull request?
Use `pyspark.sql.DataFrame`, not the classic one, in `mlutils.py`.
### Why are the changes needed?
We have connect only CI which does not even have class DataFrame. This util should work with connect DataFrame too.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
`test_pipeline` and `test_parity_pipeline` passed locally.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #55630 from gaogaotiantian/fix-mlutils.
Authored-by: Tian Gao <gaogaotiantian@hotmail.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>1 parent 6bfe0ef commit 2df302d
1 file changed
Lines changed: 6 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
29 | | - | |
| 28 | + | |
30 | 29 | | |
31 | 30 | | |
32 | 31 | | |
| |||
100 | 99 | | |
101 | 100 | | |
102 | 101 | | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
103 | 107 | | |
104 | 108 | | |
105 | 109 | | |
| |||
0 commit comments