Merge branch 'main' into aalam-SNOW-2257191-cte-join-bugfix

Author: sfc-gh-aalam

CHANGELOG.md

@@ -4,6 +4,11 @@
 
 ### Snowpark Python API Updates
 
+#### Bug Fixes
+
+- Fixed a bug that `DataFrame.limit()` fail if there is parameter binding in the executed SQL.
+- Added an experimental fix for a bug in schema query generation that could cause invalid sql to be genrated when using nested structured types.
+
 #### New Features
 
 - Added a new module `snowflake.snowpark.secrets` that provides Python wrappers for accessing Snowflake Secrets within Python UDFs and stored procedures that execute inside Snowflake.
@@ -12,14 +17,65 @@
   - `get_secret_type`
   - `get_username_password`
   - `get_cloud_provider_token`
+- Added support for the following scalar functions in `functions.py`:
+  - `array_remove_at`
+  - `as_boolean`
+  - `booland`
+  - `boolnot`
+  - `boolor`
+  - `boolor_agg`
+  - `boolxor`
+  - `chr`
+  - `decode`
+  - `div0null`
+  - `dp_interval_high`
+  - `dp_interval_low`
+  - `greatest_ignore_nulls`
+  - `h3_cell_to_boundary`
+  - `h3_cell_to_children`
+  - `h3_cell_to_children_string`
+  - `h3_cell_to_parent`
+  - `h3_cell_to_point`
+  - `h3_compact_cells`
+  - `h3_compact_cells_strings`
+  - `h3_coverage`
+  - `h3_coverage_strings`
+  - `h3_get_resolution`
+  - `h3_grid_disk`
+  - `h3_grid_distance`
+  - `h3_int_to_string`
+  - `h3_polygon_to_cells`
+  - `h3_polygon_to_cells_strings`
+  - `h3_string_to_int`
+  - `h3_try_grid_path`
+  - `h3_try_polygon_to_cells`
+  - `h3_try_polygon_to_cells_strings`
+  - `h3_uncompact_cells`
+  - `h3_uncompact_cells_strings`
+  - `haversine`
+  - `hex_decode_binary`
+  - `last_query_id`
+  - `last_transaction`
+  - `least_ignore_nulls`
+  - `nullif`
+  - `nvl2`
+  - `regr_valx`
+  - `st_area`
+  - `st_asewkb`
+  - `st_asewkt`
+  - `st_asgeojson`
+  - `st_aswkb`
 
 #### Bug Fixes
 
 - Fixed an issue where DataFrame joins would fail when CTE optimization was enabled and one of the DataFrames was created using the `DataFrame.alias()` method.
 
+
 ### Snowpark pandas API Updates
 
 #### New Features
+- Added support for `DataFrame.query` for dataframes with single-level indexes.
+- Added support for `DataFrameGroupby.__len__` and `SeriesGroupBy.__len__`.
 
 #### Improvements
 
@@ -62,6 +118,7 @@
   - `create_or_replace_dynamic_table`
 - Added a new function `snowflake.snowpark.functions.vectorized` that allows users to mark a function as vectorized UDF.
 - Added support for parameter `use_vectorized_scanner` in function `Session.write_pandas()`.
+- Added support for parameter `session_init_statement` in udtf ingestion of `DataFrameReader.jdbc`(PrPr).
 - Added support for the following scalar functions in `functions.py`:
   - `getdate`
   - `getvariable`
@@ -76,6 +133,8 @@
 
 #### Bug Fixes
 
+- Fixed a bug that `query_timeout` does not work in udtf ingestion of `DataFrameReader.jdbc`(PrPr).
+
 #### Deprecations
 
 - Deprecated warnings will be triggered when using snowpark-python with Python 3.9. For more details, please refer to https://docs.snowflake.com/en/developer-guide/python-runtime-support-policy.

docs/source/modin/supported/dataframe_supported.rst

@@ -332,7 +332,7 @@ Methods
 |                             |                                 |                                  | ``"linear"`` or ``"nearest"``, and ``method`` is   |
 |                             |                                 |                                  | ``"single"``.                                      |
 +-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+
-| ``query``                   | N                               |                                  |                                                    |
+| ``query``                   | P                               |                                  | No support for dataframes with a row MultiIndex.   |
 +-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+
 | ``radd``                    | P                               | ``level``                        |                                                    |
 +-----------------------------+---------------------------------+----------------------------------+----------------------------------------------------+

docs/source/modin/supported/groupby_supported.rst

@@ -121,6 +121,8 @@ Computations/descriptive stats
 +-----------------------------+---------------------------------+----------------------------------------------------+
 | ``last``                    | P                               | Does not support ``min_count`` parameter           |
 +-----------------------------+---------------------------------+----------------------------------------------------+
+| ``__len__``                 | Y                               |                                                    |
++-----------------------------+---------------------------------+----------------------------------------------------+
 | ``max``                     | Y                               | See ``count``                                      |
 +-----------------------------+---------------------------------+----------------------------------------------------+
 | ``mean``                    | Y                               | See ``count``                                      |

docs/source/snowpark/functions.rst

@@ -59,6 +59,7 @@ Functions
     array_position
     array_prepend
     array_remove
+    array_remove_at
     array_reverse
     array_size
     array_slice
@@ -70,6 +71,7 @@ Functions
     arrays_zip
     as_array
     as_binary
+    as_boolean
     as_char
     as_date
     as_decimal
@@ -109,6 +111,11 @@ Functions
     bitshiftright
     bitxor
     bitxor_agg
+    boolor_agg
+    booland
+    boolnot
+    boolor
+    boolxor
     build_stage_file_url
     builtin
     bround
@@ -123,6 +130,7 @@ Functions
     charindex
     check_json
     check_xml
+    chr
     coalesce
     col
     collate
@@ -180,13 +188,16 @@ Functions
     dayofmonth
     dayofweek
     dayofyear
+    decode
     degrees
     dense_rank
     desc
     desc_nulls_first
     desc_nulls_last
     div0
     divnull
+    dp_interval_high
+    dp_interval_low
     editdistance
     endswith
     equal_nan
@@ -223,12 +234,36 @@ Functions
     getdate
     getvariable
     greatest
+    greatest_ignore_nulls
     grouping
     grouping_id
     hash
+    haversine
     hex
+    hex_decode_binary
     hex_encode
     hour
+    h3_cell_to_boundary
+    h3_cell_to_parent
+    h3_cell_to_point
+    h3_compact_cells
+    h3_compact_cells_strings
+    h3_coverage
+    h3_coverage_strings
+    h3_get_resolution
+    h3_grid_disk
+    h3_grid_distance
+    h3_polygon_to_cells
+    h3_polygon_to_cells_strings
+    h3_string_to_int
+    h3_try_polygon_to_cells_strings
+    h3_cell_to_children
+    h3_cell_to_children_string
+    h3_int_to_string
+    h3_try_grid_path
+    h3_try_polygon_to_cells
+    h3_uncompact_cells
+    h3_uncompact_cells_strings
     iff
     ifnull
     in_
@@ -265,9 +300,12 @@ Functions
     kurtosis
     lag
     last_day
+    last_query_id
+    last_transaction
     last_value
     lead
     least
+    least_ignore_nulls
     left
     length
     listagg
@@ -308,8 +346,10 @@ Functions
     not_
     nth_value
     ntile
+    nullif
     nullifzero
     nvl
+    nvl2
     object_agg
     object_construct
     object_construct_keep_null
@@ -346,6 +386,7 @@ Functions
     regr_sxx
     regr_sxy
     regr_syy
+    regr_valx
     repeat
     replace
     right
@@ -380,6 +421,11 @@ Functions
     strip_null_value
     strtok_to_array
     struct
+    st_area
+    st_asewkb
+    st_asewkt
+    st_asgeojson
+    st_aswkb
     substr
     substring
     sum